Building a Lightweight C-Based Progress Bar Library Inspired by Python’s Tqdm

I was working on a project— creating a fast-bpe implementation similar to TikToken & SentencePiece tokenizer, but in C/C++, & I figured out that there was no library for this, so I created this small header-only implementation of Python’s Tqdm library in C/C++.

main header image

This blog takes you through the journey of designing and implementing the tqdm-like library in C, covering the underlying principles, key design choices, and the technical challenges faced along the way.

The Need for a Progress Bar in C Applications

Progress bars are ubiquitous in software that involves lengthy tasks — whether it’s downloading files, processing data, or compiling code. While several languages provide ready-to-use libraries, C developers often find themselves implementing custom progress indicators from scratch.

The tqdm library in Python is beloved for its simplicity: it handles formatting, rate calculation, and estimated time display seamlessly. The goal of this project was to replicate that level of functionality in C, a language known for its performance and low-level control.

Core Functionality and Features

The library was designed to include the following core features:

1. Dynamic Progress Display: A progress bar that updates in-place without printing new lines.
2. Elapsed and Estimated Time Display: The bar shows how much time has passed since the start of the operation and estimates the remaining time.
3. Rate Calculation: It calculates and displays the rate of progress, optionally scaled with SI prefixes (K, M, G).
4. Custom Descriptions and Units: Users can provide custom descriptions and units to improve clarity.
5. Cross-Platform Compatibility: The implementation is designed to work seamlessly on both Linux and Windows.

Structuring the Library

tqdm Struct

At the heart of the library is the tqdm struct, which encapsulates all the necessary information to track and display progress:

typedef struct {
  const char* desc;       // Description (e.g., "Loading...")
  bool disable;           // Flag to disable the progress bar
  const char* unit;       // Unit of measurement (e.g., "iters/sec")
  bool unit_scale;        // Flag to scale units (e.g., K/M/G)
  int total;              // Total number of iterations
  int current;            // Current progress
  int skip;               // Step size for updating progress
  double start_time;      // Start time in seconds
  int rate;               // Update rate in Hertz
} tqdm;

This struct is designed to be flexible enough to handle a wide range of use cases, from simple iteration counters to complex operations with custom units.

Initialization and Update Functions

1- init_tqdm

void init_tqdm(tqdm* bar, const char* desc, bool disable, const char* unit, bool unit_scale, int total, int rate);

Purpose: Initializes the tqdm progress bar with user-defined parameters.

Parameters:

• tqdm* bar: Pointer to the progress bar struct that will be initialized.
• const char* desc: A string description (e.g., "Loading...") to be displayed alongside the progress bar.
• bool disable: A flag indicating whether the progress bar should be disabled. If true, no updates will be printed.
• const char* unit: Unit of measurement for the progress (e.g., "iters/sec").
• bool unit_scale: When set to true, the rate is scaled using SI prefixes (K/M/G).
• int total: The total number of iterations to be completed.
• int rate: Update rate in Hertz, specifying how frequently the progress bar should be refreshed.

Functionality: This function sets the initial values of the tqdm struct, including starting the timer by calling get_time(). It ensures that total iterations and rate have valid values (non-negative). This setup is essential for the correct display and calculation of progress.

2- update_tqdm

void update_tqdm(tqdm* bar, int increments, bool close);

Purpose: Updates the progress bar by a specified number of increments.

Parameters:

• tqdm* bar: Pointer to the progress bar struct to be updated.
• int increments: The number of steps to increment the current progress.
• bool close: If true, the progress bar is finalized (printed one last time with a newline).

Functionality: The function increments the current progress count by increments. If the current count exceeds the total, it is capped at the total. It then calls print_tqdm() to refresh the display. If close is true, a newline is printed at the end to finalize the progress bar.

Key Considerations:

• Handles cases where the total progress might be zero.
• Ensures that the display remains smooth and accurate by updating at a controlled rate.

3- print_tqdm

void print_tqdm(tqdm* bar, bool close);

Purpose: Prints the progress bar to the console.

Parameters:

• tqdm* bar: Pointer to the progress bar struct.
• bool close: If true, a newline is printed after the progress bar.

Functionality: This function performs several tasks:

1. Calculates elapsed time: Uses get_time() to determine how long the operation has been running.
2. Computes progress percentage: Divides the current count by the total to get the completion percentage.
3. Formats time and rate: Calls HMS() to format elapsed and remaining time, and SI() to scale the rate if needed.
4. Displays the bar: Constructs a visual representation of the progress using = and `` characters, showing the percentage, current/total count, elapsed time, remaining time, and rate.

Utility Functions

1- get_time

static double get_time();

Purpose: Returns the current time in seconds with high precision.

Functionality: This function uses clock_gettime() on Linux to obtain a monotonic clock value (one that increases steadily without being affected by system time changes). It converts the result to seconds with nanosecond precision.

Cross-Platform Consideration: For Windows, an equivalent high-resolution timer such as QueryPerformanceCounter() would be used, wrapped in #ifdef directives to ensure portability.

2- HMS

void HMS(double seconds, char* output, size_t buffer_size);

Purpose: Converts a time duration in seconds to a human-readable string in the format HH:MM:SS.

Parameters:

• double seconds: Time duration to be formatted.
• char* output: Buffer to store the formatted string.
• size_t buffer_size: Size of the output buffer.

Functionality: The function calculates hours, minutes, and seconds from the total duration. It then uses snprintf() to write the formatted string into the output buffer.

Example: If seconds = 3661, the output would be 01:01:01.

3- SI

void SI(double value, char* output, size_t buffer_size);

Purpose: Scales a numeric value using SI prefixes (K, M, G, etc.) and formats it as a string.

Parameters:

• double value: The numeric value to be scaled.
• char* output: Buffer to store the formatted string.
• size_t buffer_size: Size of the output buffer.

Functionality: The function iteratively divides the value by 1000 while incrementing the prefix index until the value is less than 1000 or the highest prefix (Y) is reached. It then formats the scaled value and prefix into the output buffer.

Example: If value = 1500, the output would be 1.50k.

Specialized Functions

1- init_trange

void init_trange(tqdm* bar, int n, const char* desc, bool disable, const char* unit, bool unit_scale, int rate);

Purpose: A convenience function for initializing a range-based progress bar.

Parameters: Similar to init_tqdm, but with n specifying the total number of iterations directly.

Functionality: This function wraps around init_tqdm() to simplify initialization for use cases where only a total count and basic description are needed.

2- close_tqdm

void close_tqdm(tqdm* bar);

Purpose: Disables the progress bar, effectively stopping any further updates.

Functionality: Sets the disable flag to true, ensuring that no further output is generated for the progress bar.

Recursive Utilities

1- dfs

void dfs(void* x, PrettyCacheEntry* cache, size_t cache_size, void** (*srcfn)(void*));

Purpose: Performs a depth-first search on a data structure, caching visited nodes.

Parameters:

• void* x: Starting node.
• PrettyCacheEntry* cache: Array of cache entries for visited nodes.
• size_t cache_size: Size of the cache.
• void** (*srcfn)(void*): Function pointer to retrieve children of a node.

Functionality: The function recursively traverses the data structure, updating the cache to mark visited nodes. This ensures that cyclic structures are handled correctly without infinite loops.

running test cases

Handling Edge Cases

Building a robust library means anticipating edge cases and handling them gracefully. Some of the key considerations included:

1. Zero or Negative Totals: If the total number of iterations is zero or negative, the progress bar simply displays elapsed time without estimating the remaining time.
2. Disabled Progress Bar: When the disable flag is set, all update and display operations are bypassed to minimize overhead.
3. Floating-Point Precision: Time and rate calculations use double-precision floating-point arithmetic to maintain accuracy over long durations.

Cross-Platform Support

To ensure compatibility across different operating systems, the library uses clock_gettime on Linux and QueryPerformanceCounter on Windows for high-resolution timing. The code is wrapped in #ifdef directives to select the appropriate implementation at compile time.

Compiling and Using the Library

The library can be compiled into a shared object (.so) file on Linux or a dynamic-link library (.dll) file on Windows:

# Linux
g++ -shared -fPIC -o libtqdm.so tqdm.cpp

# Windows
g++ -shared -o libtqdm.dll tqdm.cpp

Once compiled, the library can be linked to any C project. Developers can include tqdm.h in their source files and link against the compiled library.

Conclusion

This project demonstrates that with careful design and attention to detail, it’s possible to create a lightweight, efficient progress bar library in C that rivals the functionality of Python’s tqdm. While the library might not yet cover every feature of its Python counterpart, it provides a solid foundation for future enhancements.

Future work could include adding support for multi-threaded environments, customizable bar styles, and more granular control over display formatting.

The complete source code for the library is available here. If you have feedback or suggestions, feel free to contribute!

PS: this blog was written by chatgpt, I just made a few modifications :-D