UUIDs in Python: Use Cases & How to Speed Them Up ⚡

Use UUIDs effectively: tools, techniques, and knowing when to switch

UUIDs with Python — slide by author, AI generated image inside, also by author.

A UUID stands for Universally Unique Identifier. It’s a 128‑bit value designed to identify information across systems without a central authority. Systems use UUIDs in databases, network protocols, and distributed caches.

UUIDs follow the de facto standard defined in **RFC 4122. They’re popular in databases and distributed systems because they can be generated independently, from a function call, without needing a centralized counter or global coordination**.

Python provides a built‑in uuid module, which supports versions 1, 3, 4, and 5. However, it lacks support for version 7 (time‑ordered), and it performs relatively slowly when generating large volumes. The Python ecosystem offers other libraries that help you deal with those limitations; you'll discover some of them in this article.

First, you’ll find a concise overview of the different UUID versions and their typical use cases (skip it if you’re just here for the Python code). Then, we’ll walk through Python examples using both the standard library and faster third-party alternatives.

Let’s go!

Understanding UUIDs

What is a UUID?

A UUID is an identifier that ensures uniqueness across time, space, and systems, without needing coordination. It achieves this by using mathematical functions that generate values randomly selected from a very large space (2¹²⁸ possible values). The chance of generating the same value twice is so low it’s considered negligible.

• Version 4 uses cryptographically secure randomness to pick from that massive space.
• Version 1 encodes a timestamp, a node ID (like a MAC address or a random 48‑bit number), and a sequence counter to guarantee uniqueness across machines and time.

UUIDs are usually shown as 36-character hex strings with hyphens, like 123e4567-e89b-12d3-a456-426614174000. You can generate them in any language or system, for example, using Python.

So when you generate a UUID, you’re very probably getting a truly unique value, and one you can safely use as an ID.

UUID Versions

UUIDs come in multiple versions, each designed for specific needs. Later versions often improve on earlier ones, such as version 7, which enhances version 4 by enabling sortability. Here’s a quick overview:

• Version 1 (time‑based) combines a high-precision timestamp, a node ID (MAC address or random), and a sequence counter to ensure uniqueness across machines and time. Older systems used real MACs (potential privacy leak), but modern libraries typically use random IDs. It can misbehave if clocks drift backward.
• Version 2 (DCE Security) extends v1 with embedded POSIX user/group IDs. It’s rarely used today and unsupported in Python.
• Version 3 (namespace + MD5) hashes a namespace and name string using MD5. It’s deterministic: same input, same UUID. It’s useful for consistent identifiers, though MD5 is no longer cryptographically secure.
• Version 4 (random) uses 122 bits of secure randomness. It’s the default in many libraries due to simplicity and excellent collision resistance.
• Version 5 (namespace + SHA‑1) works like v3 but with SHA‑1, offering stronger (but still limited) collision resistance. Also deterministic.
• Version 6 (reordered time) restructures v1 by placing the timestamp first, improving index locality and insert performance in time‑sorted databases.
• Version 7 (time‑ordered) merges a 48-bit Unix timestamp with 80 bits of randomness, making it sortable and safe from collisions. It’s ideal for logs and time-series data. It’s gaining popularity over v4.
• Version 8 (custom) lets you define your own structure. It gives flexibility but leaves uniqueness guarantees up to you.

Let’s take a look at some practical use cases for UUIDs.

Use Cases

Decentralized Identifiers: UUIDs are ideal for decentralized systems because they don’t rely on a central authority (like a database or coordinator).

Any node can generate a UUID independently, and it remains unique, even across different machines or services. This makes UUIDs perfect for microservices, distributed caches, and peer-to-peer networks, where coordination is costly or impossible.

In distributed NoSQL databases (e.g., MongoDB, Cassandra), UUIDs allow records to be created across shards or nodes without conflicts.

OLAP Systems:

OLAP workloads involve batch inserts into massive tables. Sequential IDs create hotspots on clustered indexes, slowing down inserts.

To improve performance, OLAP systems often skip primary keys entirely. UUIDs offer an interesting compromise: globally unique identifiers that don’t require coordination. They are faster to write, but are harder to index and take more space compared to integer unique keys (where generation needs to be centralized).

Another advantage: you can generate the UUIDs outside the database, for example, you can create them in Python scripts, Airflow tasks, or Kafka streams. This is useful when your DB doesn’t support UUIDs directly or when generating IDs before ingestion improves throughput.

Horizontal Scaling:

In distributed systems like transaction logs and event-sourced architectures, UUIDs eliminate the need for a central ID generator.

Auto-increment keys require coordination, which becomes a bottleneck at scale. UUIDs let each node generate identifiers independently, preserving performance under high load.

Version 4 offers collision-resistant IDs for distributed caches and logs. Version 7 adds sortability, which is critical in event streams where chronological order matters, like for state reconstruction or time-based analytics.

AI and Data Pipelines:

In ML pipelines, UUIDs can tag experiment runs, dataset snapshots, and model versions. Version 7 UUIDs, being time‑ordered, make it easy to sort and reproduce experiments by generation time.

In vector databases, UUIDs are often used to identify embeddings and their metadata, or to update/delete vectors by ID.

Unique File Names and Storage Keys:

UUIDs are widely used to generate unique filenames or keys, especially in cloud and parallel systems.

Examples include:

• S3 object keys (data_13f3e2d2-7769-4b10-879e-a011e5f3d0e9.json)
• Naming uploaded images or temporary files
• Identifying artifacts in pipelines like Apache Airflow or Taipy

A screenshot of a VS Code Document Explorer. The project shows lots of JSON files with unique names generated with a UUID. Screenshot by author.

By avoiding filename collisions, UUIDs allow concurrent writes without coordination.

Challenges and Limitations

UUIDs come with specific challenges and limitations. Here are key considerations to help you decide if they’re right for your project:

• Size Overhead: UUIDs are much larger than integer keys. This increases storage usage and can bloat indexes, especially in large tables where space efficiency matters. The problem is worse when UUIDs are stored as CHAR or VARCHAR, which consume more space and index poorly compared to binary formats.
• Index Fragmentation: Random UUIDs (like v4) can fragment B-tree indexes. Their scattered distribution causes frequent page splits and higher I/O during inserts. Over time, this degrades write performance. A better option for ordered inserts is UUIDv7, which preserves randomness while improving index locality.
• Overkill for Small Systems: For systems with few records or limited concurrency, UUIDs may add unnecessary complexity. Simpler IDs (like auto-incremented integers) are more efficient and easier to debug.
• RNG Quality and UUIDv4 Collisions: While UUIDv4 offers ~2¹²² unique possibilities, its reliability depends on a strong random number generator. A weak or predictable RNG can increase the (otherwise negligible) risk of collisions.

In the next section, we will look at the practical aspects of generating these various UUID versions within Python and assess the performance characteristics of different libraries designed for this purpose.

Implementing UUIDs in Python

This section shows you how to generate UUIDs and ULIDs in Python, compare performance, and encode identifiers into compact, URL‑safe formats.

2.1 Python’s Standard Library (uuid)

The built‑in uuid module lets you generate v1, v3, v4 and v5 UUIDs without external dependencies. It's pure Python, so performance isn’t great, but it’s enough for most use cases unless you’re generating UUIDs in bulk. Note that it doesn't support newer versions like UUIDv7.

Generation is straightforward: import the module and call the version you need (like uuid.uuid4()). Versions 3 and 5 require two inputs:

• A namespace, which is a predefined UUID (such as uuid.NAMESPACE_DNS, NAMESPACE_URL, NAMESPACE_OID, or NAMESPACE_X500)
• A name, which is any string

These two values are hashed together, using MD5 for v3 and SHA‑1 for v5. This means if you pass the same name and namespace again, you’ll get the same result. You typically use NAMESPACE_DNS when you're generating UUIDs for domain-like names (usernames, emails, URLs), but you can also define your own custom namespace UUID. The string itself can be as long as needed, but longer inputs will slightly slow down hashing. Encoding isn’t required, but it’s common to pass plain strings directly unless you're working with binary data.

Here’s how all this looks:

import uuid

print("uuid 1:", uuid.uuid1())
print("uuid 3:", uuid.uuid3(uuid.NAMESPACE_DNS, "Hello, how you do?"))
print("uuid 4:", uuid.uuid4())
print("uuid 5:", uuid.uuid5(uuid.NAMESPACE_DNS, "Hello, how you do?"))
# uuid 1: 3f485ede-5846-11f0-ae74-04d3b0d1c8fb 
# uuid 3: 47981fdb-ea35-39a6-940a-98904125b7e1 
# uuid 4: 81107f0e-3803-4dc8-a6a9-06837366c696 
# uuid 5: 99bf4c71-0980-5f0a-9209-d6f8f0d43810

All UUIDs have the same representation and the same length. The library allows for output of the UUID in several formats. For example, you can output a hexadecimal form (it removes the dashes), or a URN form using .hex and .urn respectively:

u = uuid.UUID("123e4567-e89b-12d3-a456-426614174000")
print(u.hex) # '123e4567e89b12d3a456426614174000'
print(u.urn) # 'urn:uuid:123e4567-e89b-12d3-a456-426614174000'

Note that these generate str type objects. The hex output can be good to create unique file names, for example. However, if we want to store data efficiently in a database, we'll need other formats. Thankfully, we can also output data as int or bytes values:

print(uuid.uuid4().int) # 37957867547679550666636220158363137819
print(uuid.uuid4().bytes) # b'\xcev\xe5\xb5#\x9bAD\xb8\x14\x7fp\x97\x9d\xc8\x03'

But wait a minute! That integer number is very big, in fact, it’s bigger than a BIGINT: DBMSs usually can't handle numbers that big! However, most DBMS can handle bytes this size; in that case, you can transform the data to bytes with Python before inserting it into a database field of type bytes.

Some databases, such as PostgreSQL (from version 17), have dedicated UUID types. In that case, you can insert string values from your Python structures, and the DBMS will store (and display!) them efficiently.

To summarize: If you need to create a unique file name, stick to a hex string. If you need to insert the data in a database, look if you have a UUID data type; if not, prefer byte types over VARCHAR. As a last resort, you can fall back on VARCHAR, which is universally supported but slower to index and less space-efficient.

Let’s now take a look at some Python alternatives to the built-in library.

UUID6

[uuid6](https://pypi.org/project/uuid6/) is a Python-written library that gives access to UUID versions 6, 7, and 8. This library is well established, but since it's written in Python, it's also slow. You use it like the regular uuid library:

import uuid6

my_uuid = uuid6.uuid7()
print(my_uuid)
# 01980614-9653-72da-b843-33460f523590

We mention this one because it’s well known; however, you could prefer the following one:

uuid‑utils (Rust‑backed)

[uuid‑utils](https://pypi.org/project/uuid-utils/)is a Python library backed by Rust, a low-level programming language. It supports UUID versions 6, 7, and 8, like the previous one, and also versions 1, 3, 4, and 5, like the one from Python's standard library.

It provides a unified API for all UUID types, and you can import and use it exactly like you would use the standard one:

import uuid_utils as uuid

print("uuid 4:", uuid.uuid4())
# uuid 4: 81107f0e-3803-4dc8-a6a9-06837366c696

This library offers a significant performance advantage, generating UUIDs approximately 10 times faster than Python-based libraries.

To demonstrate this, I conducted a basic benchmark comparing it against Python’s uuid and uuid6 libraries. The test involved generating 100,000 UUIDs per iteration, running 50 iterations for each library, and then averaging the results. While the benchmark does include the loop generation time (approximately 0.02 microseconds on my machine), this overhead applies equally to both sets of results.

You can review the full benchmark tests in this notebook.

The results for uuid-utils vs. uuid are displayed in the image below (in microseconds):

Chart comparing running time to generate UUIDs V1 to V5 with Python’s uuid’s library, vs the rust-backed uuid-utils library. Results are in microseconds. Chart generated by author, using Seaborn.

As you can see, the Rust implementation is way faster. Also, the Python implementation has much more variability. The Python uuid library has higher variability because it's interpreted code that gets interrupted by garbage collection and creates many temporary objects, while the Rust-based uuid-utils runs as pre-compiled machine code with predictable memory usage patterns.

You can see more details in the following table:

A table comparing Python’s standar uuid library vs uuid-utils. Generated by author, using Great Tables.

I also benchmarked uuid6 vs. uuid-utils for UUID V7. The results are similar; in this case, the difference seems bigger (but version 7 is harder to calculate, so that seems coherent):

Chart comparing running time to generate UUID v7 with Python-backed uuid6’s library, vs the rust-backed uuid-utils library. Results are in microseconds. Chart generated by author, using Seaborn.

Let’s now take a look at a different approach to generating UUIDs with Python when using large tabular data!

UUIDs in DuckDB (C++ Backed)

DuckDB is another project that’s coded in a low-level language (C++). It provides an API for many programming languages, including Python. DuckDB allows for in-memory analytical operations, and you can use to query files (CSVs, parquet…) directly using SQL. DuckDB is fast.

DuckDB is fast. Duck DB is great. DuckDB has a built-in UUID data type. DuckDB generates UUIDs. However, DuckDB only makes sense if you want to analyze tabular data from a DataFrame or directly from a file.

To query a file with DuckDB, you can use the DuckDB SQL function read_csv; DuckDB also has uuidv4() and uuidv7() functions. In the example below, I created a SQL query that selects all columns from a CSV file, and also creates a UUID column and fills it for all rows:

query_v4 = f"""
    SELECT
    *,
    uuidv4() AS uuid_v4
    FROM read_csv('{csv_file_path}');
"""

duckdb_df_with_uuid = duckdb.sql(query_v4)

The file that’s referenced in csv_file_path has 1 million rows. Executing the whole query takes (on my computer!) between 0.1 and 0.2 seconds, that'd be between 0.1 and 0.2 microseconds per row. That seems to be 4-5 times faster than generating them with the Rust implementation. Super fast!

Note that this includes the time to read the CSV file! In our previous benchmarks, we didn’t read any files, but we had loop overhead. So the comparison is not 100% equivalent, but the use cases are also different. You can only use DuckDB with tabular data; this method doesn’t serve all use cases: for example, you can’t use this for file naming, or to create a list with UUIDs (you always can, but it wouldn’t be practical).

Also, the fact that DuckDB has a built-in UUID data type means that it can handle it efficiently: in terms of storage, in terms of computation, and in terms of passing the values to a different DBMS should you be using it in a pipeline.

Let’s now discuss when to use which.

When to Choose Which

The Python native implementation has one big advantage over its “competitors”: You don’t need to install any other library. If you don’t need unsupported versions (version 6, 7, and 8), and speed isn’t an important requirement, it may be the right choice.

If your workflow uses DuckDB, then you’re a lucky person, since its native implementation (for recent versions of the library!) is the fastest option (that I know of). If you don’t use DuckDB, but you think you could, then try it out, there are plenty of other good reasons to use it for your analytical workflows.

**uuid-utilsis also very fast (about 10x times faster than the standard library) and provides access to UUID versions 6, 7, and 8. If you can afford adding an (overall small) dependency to your project, and you're not using DuckDB, this is probably your best option. For example, you can use it to speed up unique file naming or to generate UUIDs fastly in DataFrames if DuckDB isn't an option**.

The uuid6 library seems less useful: if you need to add a dependency to your project because Python's native library doesn't support UUID v7, then you might as well take the extra speed provided by uuid-utils.

Conclusion

UUIDs are useful for generating unique identifiers across distributed systems, databases, and pipelines, with no central coordination. They’re a solution to consider in environments where scalability, decentralization, and independence matter.

In this article, we walked through the theory behind the different UUID versions, their practical tradeoffs, and how to use them in Python with both standard and third-party libraries. We also benchmarked performance and explored how tools like DuckDB can provide an even faster path when working with tabular data.

But UUIDs aren’t always the right solution.

If your system is simple, or if insertion performance and tight indexing matter more than global uniqueness, traditional integer IDs might be a better fit. Don’t reach for UUIDs just because they seem “cool” or “future-proof”: they introduce complexity, increase storage size, and can hurt index locality (unless you use sortable versions like v7).

Also, UUIDs aren’t the only unique ID generator:

• ULIDs are lexicographically sortable and use Base32 encoding for shorter strings. They are great for logs and URLs, but they’re not a standard (yet?).
• KSUIDs, designed by Segment, combine sortability with randomness.
• Microsoft GUIDs are essentially UUIDs with different generation semantics (sometimes encoded in little-endian).
• Snowflake IDs, popularized by Twitter, which embed a timestamp, machine ID, and sequence.
• Even Bitcoin-style hashes, like SHA-256-based transaction IDs, offer a kind of identity based on content, not just uniqueness.

To summarize: UUIDs are versatile, but they’re not a one-size-fits-all solution. Choose the right identifier strategy based on your needs: performance, sortability, debuggability, and storage constraints all matter. And when you do use UUIDs, also select the right tools: DuckDB for fast in-memory analytics, uuid-utils for fast Python generation, Python's native implementation for simple dependency management, and byte-level storage to keep database performance.

Additional Resources

• Andrew Atkinson on OLTP systems and UUID limitations: A well-rounded critique of UUIDs in high-throughput transactional databases.
• A brief history of the UUID: Segment’s detailed overview of UUID evolution (up to v4) that introduces KSUID as the next-gen unique ID system. Since Segment developed KSUID, their bias toward it is understandable, but it remains a good read.
• IN FRENCH: ULID vs UUID in web development: An insightful comparison of ULIDs and UUIDs, written in French.
• Wikipedia: Universally unique identifier: A comprehensive reference covering all UUID versions, variants, and applications. Also comes with dozens of other reference links!

Thank you for reading!

[embed]Python concepts Edit descriptionericnarro.medium.com

If you like my content and want to connect:

👉 Check all my other accounts

☕ You can buy me a coffee