Advanced Python: Decorators, Async, Generators, the GIL and More

Advanced Python: decorators, async, generators, the GIL, metaprogramming and more practical concepts.

Ammar Raza · 2026-04-06 03:17 · 0 claps · 7.8 min read

#advanced-python-topic #python-internals #python-decorators #python-async #python-generators

Advanced Python: Precise Explanation Of Concepts That Make You Better

One of the reasons that people love Python is the relative ease of creating useful programs without wrestling with its syntax.

But the deeper you go, the more you realize that writing efficient Python programs is about understanding how the language behaves, how to structure code well, how to manage resources safely and how to think about performance and concurrency.

That is where advanced Python topics start to matter. Concepts like decorators, context managers, generators, async programming, serialization, metaprogramming, CPython, the GIL and so many more concepts, are not just “nice to know”. They help explain why Python works the way it does and how experienced engineers write cleaner, safer and more effective code.

This article gives a very brief overview of some of those ideas in a practical way. What they are, why they matter and where they show up in real code.

Advanced Python is really about better design

The phrase “advanced Python” can sound intimidating, but most of these topics are not about being clever.

They are about writing code that is:

easier to extend
more reusable
safer around resources
more memory efficient
better aligned with how Python actually runs

That is the real value. Advanced Python is less about tricks and more about engineering. With that being said, lets start looking into some of those concepts now.

Decorators

Decorators let you wrap a function, method or class and change its behavior without editing its source code directly. Some common use cases are logging, validation, timing and caching.

import time

def timer(func):
    def wrapper(*args, **kwargs):
        start = time.time()
        result = func(*args, **kwargs)
        print(f"{func.__name__} took {time.time() - start:.4f}s")
        return result
    return wrapper
@timer
def slow_function(a, b):
    time.sleep(1)
    return a + b

Decorators are powerful because they separate the main logic from reusable surrounding behavior.

Metaprogramming

Metaprogramming is code that generates, modifies or adapts code behavior programmatically. Its use cases are monkey patching and creating or modifying classes at runtime.

def create_model_class(name, fields):
    return type(name, (object,), fields)

User = create_model_class("User", {"role": "admin"})

This becomes useful in frameworks, plugin systems, ORMs and libraries that need to adapt based on configuration.

Jinja templates

Jinja is a template engine that takes tokenized strings and fills in user supplied values. Common use case is dynamic content generation such as web output and messages.

# Before using Jinja, you need to install it. You can install Jinja using pip:
pip install Jinja2

# Creating a Jinja Template: First, create a template file with the .jinja or .html extension. For example, let's create a template file named template.jinja:
<!DOCTYPE html>
<html>
<head>
    <title>Welcome</title>
</head>
<body>
    <h1>Hello, {{ name }}!</h1>
</body>
</html>
from jinja2 import Template
# Load the template from the file
with open('template.jinja') as file:
    template = Template(file.read())
# Render the template with data
rendered_template = template.render(name='John Doe')
# Print the rendered template
print(rendered_template)

The value of templating is clean separation between structure and data.

Context managers

Context managers help manage resources safely by guaranteeing setup and cleanup around a block of code. Files and database connections are common practical examples.

with open("file.txt", "w") as f:
    f.write("Hello")

You can also build your own:

class FileManager:
    def __init__(self, filename):
        print("Entering the context initialization")
        self.filename = filename

  def __enter__(self):
          print(f"Entering the context and opening {self.filename}")
          self.file = open(self.filename, 'w')
          return self.file
      def __exit__(self, exc_type, exc_val, exc_tb):
          print(f"Exiting the context and closing {self.filename}")
          self.file.close()

with FileManager('file.txt') as file:
    print(f"Writing to {file.name}")
    file.write('Hello, World!')
    # Perform other operations with the file

The enter() method is called when the with block is entered. It performs any necessary setup actions, and it can return a resource or any value that will be assigned to the target variable (resource in the example).

The exit() method can return a Boolean value to control the propagation of exceptions. Returning True suppresses any exceptions raised within the block, while returning False allows exceptions to propagate as usual.

This is one of Python’s cleanest patterns for reliable resource handling.

Lambda functions

A lambda is a small anonymous inline function. Lambda is best for short, single line logic, especially with map, filter, and similar patterns.

nums = [1, 2, 3]
squared = list(map(lambda x: x * x, nums))

add = lambda x, y, z=0: x + y + z

print(add(2, 3))     # Output: 5
print(add(2, 3, 4))  # Output: 9

Lambdas are useful when the logic is tiny. If they become hard to read, a normal function is usually better.

Advanced data structures

Python offers richer data structures than just lists and dicts. There are also defaultdict, OrderedDict, Counter, namedtuple, heap, deque, stack style usage and tree data structures, each with their own use cases.

from collections import Counter, deque

print(Counter(["python", "sql", "python"]))
dq = deque([1, 2, 3])
dq.appendleft(0)

These structures matter because different workloads care about different access patterns and performance tradeoffs.

Optimization

Optimization in Python can mean improving speed, reducing memory usage or avoiding repeated work. Caching, memoization and Cython to some extent are most common methods for optimization.

from functools import lru_cache

@lru_cache(maxsize=None)
def fib(n):
    if n < 2:
        return n
    return fib(n - 1) + fib(n - 2)

A lot of optimization is not about making code “faster” in a vague sense. It is about removing unnecessary repeated work.

CPython and the interpreter

A Python interpreter is a program that executes Python code. It reads and interprets the Python source code, converts it into machine-readable instructions, and executes those instructions to produce the desired output.

CPython is the default and most widely used implementation of Python. It is written in C and is the reference implementation of the Python language. Here’s an overview of how CPython works:

Lexing and Parsing: CPython starts by reading the Python source code and breaking it down into tokens through a process called lexing. It identifies keywords, identifiers, operators, and other elements in the code. Then, the tokens are parsed to build an Abstract Syntax Tree (AST), representing the structure and relationships between the different components of the code.

Bytecode Compilation: Once the AST is constructed, CPython compiles it into bytecode, which is a lower-level representation of the code that is more easily executed by the interpreter. The bytecode consists of a series of instructions specific to the Python virtual machine.

Interpreter and Virtual Machine: CPython’s interpreter executes the bytecode instruction by instruction. It loads the bytecode into the Python virtual machine, which is responsible for managing the execution of the code. The virtual machine includes components like the stack for storing values and frames for managing function calls and local variables.

Execution: The interpreter fetches each bytecode instruction, executes it, and moves on to the next one. The instructions can involve operations like variable assignments, function calls, control flow statements (if, for, while), and more. CPython performs the necessary operations to evaluate expressions, manipulate objects, and execute the logic of the program.

Object Model and Memory Management: CPython has a robust object model that represents data and code as objects with associated types and behaviors. It manages memory allocation and deallocation through a garbage collector, which automatically reclaims memory that is no longer in use to prevent memory leaks.

Interaction with C Extensions: CPython can also interact with C extensions and modules. It provides an interface for integrating C code, allowing developers to write high-performance modules or extend Python with functionality that may require low-level operations.

The GIL

The Global Interpreter Lock (GIL) is a mechanism in CPython, the default and most widely used implementation of Python, that ensures only one thread executes Python bytecode at a time. It acts as a lock that prevents multiple native threads from executing Python bytecodes simultaneously. This has implications for multi-threaded Python programs, as it can impact concurrency and parallelism.

In CPython, the GIL is necessary to protect shared data structures, such as Python objects, from simultaneous access and potential data corruption. However, it restricts true parallelism within a single Python process, as only one thread can execute Python bytecodes at any given time. This means that even with multiple threads, only one thread can be actively executing Python code, while other threads may be waiting to acquire the GIL.

Let’s consider an example that illustrates the impact of the GIL where the actual output will be different than the expected output:

import threading
import time

counter = 0
def increment_many(n):
    global counter
    for _ in range(n):
        temp = counter
        time.sleep(0)   # hint to scheduler: switch threads here
        counter = temp + 1
num_threads = 10
increments_per_thread = 1000
threads = [
    threading.Thread(target=increment_many, args=(increments_per_thread,))
    for _ in range(num_threads)
]
for t in threads:
    t.start()
for t in threads:
    t.join()
expected = num_threads * increments_per_thread
print("Expected:", expected)
print("Actual:  ", counter)

The solution to prevent such a race condition is using lock, like this:

import threading
import time

counter = 0
lock = threading.Lock()
def increment_many(n):
    global counter
    for _ in range(n):
        with lock:
            temp = counter
            time.sleep(0)
            counter = temp + 1
num_threads = 10
increments_per_thread = 1000
threads = [
    threading.Thread(target=increment_many, args=(increments_per_thread,))
    for _ in range(num_threads)
]
for t in threads:
    t.start()
for t in threads:
    t.join()
print(counter)

Multithreading

Multithreading is especially useful for I/O-bound workloads, such as waiting for responses while other work continues.

import threading, time

def task(name):
    time.sleep(2)
    print(name)
t1 = threading.Thread(target=task, args=("task1",))
t2 = threading.Thread(target=task, args=("task2",))
t1.start(); t2.start()
t1.join(); t2.join()

Threads are often best when the workload spends time waiting.

Multiprocessing

Multiprocessing uses separate processes instead of threads. It as better for CPU intensive workloads because each process gets its own resources.

from multiprocessing import Process

def task():
    print("running")
p1 = Process(target=task)
p2 = Process(target=task)
p1.start(); p2.start()
p1.join(); p2.join()

The usual rule is simple. Use multithreading for I/O bound work and use multiprocesseing for CPU bound work.

Asynchronous programming

Async programming is another form of concurrency, especially useful for tasks involving waiting on network or file operations. It is a way to reduce runtime by moving I/O heavy tasks into background.

A function designed as asynchronous function is called “coroutine”.

import asyncio

async def task(name, delay):
    print(f"Task {name} started")
    print(f'Task {name} before await call')
    await asyncio.sleep(delay)
    print(f'Task {name} after await call')
    print(f"Task {name} completed")
# Asynchronous execution
async def main():
    await asyncio.gather(
        task("A", 2),
        task("B", 2),
        task("C", 1),
        task("D", 1)
    )
asyncio.run(main())

Async shines when you have many tasks that spend a lot of time waiting.

Generators

Generators produce values one at a time instead of building an entire collection in memory. Their role shines in memory savings and large dataset iteration.

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

# Using the generator
fib_gen = fibonacci()
for _ in range(10):
    print(next(fib_gen))

This is one of the simplest and best tools in Python for memory efficient iteration.

Serialization and deserialization

Serialization converts objects into storage friendly or transmission friendly formats. Deserialization reconstructs them back into program usable objects.

Pickle is a built-in python module that is commonly used to perform this operation in Python.

import pickle

# Object to be pickled
data = {
    'name': 'Alice',
    'age': 30,
    'city': 'New York'
}
# Pickling the object to a file
with open('data.pickle', 'wb') as file:
    pickle.dump(data, file)
# Unpickling the object from a file
with open('data.pickle', 'rb') as file:
    loaded_data = pickle.load(file)
# Print the unpickled object
print(loaded_data)

This shows up constantly in APIs, distributed systems, caching and persistence.

What changed once I understood these topics

Once I understood these concepts, it became clear that each of them has a well defined purpose and are not just overlapping buzzwords.

Final thoughts

Advanced Python is not about showing off.

It is about understanding the concepts that make code cleaner, more scalable, more memory aware, and easier to reason about. Decorators, metaprogramming, Jinja, context managers, lambda, data structures, optimization, CPython, the interpreter, the GIL, multithreading, multiprocessing, async, generators and serialization all contribute to that bigger picture.

The more you understand them, the more Python starts to feel less like a beginner language and more like a serious engineering tool.

메타데이터

post_id: d46f4d22bbe0
slug: advanced-python-decorators-async-generators-the-gil-and-more-d46f4d22bbe0
url: https://medium.com/@ammarraza_2442/advanced-python-decorators-async-generators-the-gil-and-more-d46f4d22bbe0
canonical_url: https://medium.com/@ammarraza_2442/advanced-python-decorators-async-generators-the-gil-and-more-d46f4d22bbe0
author_url: https://medium.com/@ammarraza_2442
status: ok
fetched_at: 2026-08-25 23:37:46