Python Iterators and Iterable Objects

From zip() to Building Your Own Iterators

Python Iterators and Iterable Objects | Slide by Author with and AI generated image inside

In Python, an iterator is an object that lets you loop (iterate) over a certain “container”, such as lists, sets, tuples, or dictionaries, one element at a time, without loading all elements into memory at once. This makes iterators memory-efficient, especially when dealing with large datasets. The zip() function is an example of an iterator function. From a technical point of view, an iterator is an object that implements two methods: __iter__() and __next__().

Everything in Python is an object: lists, dictionaries, tuples, functions… Many of these are iterable objects, which means they can be looped over. This is the foundation behind for loops and comprehensions.

Under the hood, not all iterables are the same. Some objects are just containers that can be turned into an iterator. Others are also iterators and produce values one at a time, lazily, and remember where they left off. Understanding this distinction is useful if you’re working with large datasets, streams, or want to create clean and efficient interfaces.

In this article, we’ll explore how iteration works in Python. We’ll look at common built-in iterators like zip() and enumerate(), and then we'll write our own using classes and generator functions.

Iterating and Iterators

Let’s start simple. You’ve probably used a for loop, like this:

for item in [1, 2, 3]:
    print(item)

But what happens under the hood? Python does this instead:

it = iter([1, 2, 3])
print(next(it))  # 1
print(next(it))  # 2
print(next(it))  # 3
print(next(it)) # This outputs StopIteration error

When Python loops, it calls iter() on the object, then repeatedly calls next() on the result until it raises StopIteration.

Iterable vs Iterator

All iterators are always iterables, but an iterable may or may not be an iterator. Here’s the difference:

• An iterable is any object that can return an iterator. It has an __iter__() method.
• An iterator is an object that has a __next__() method and returns itself when __iter__() is called.

Let’s check this with a list:

mylist = [10, 20, 30]
it = iter(mylist)

print(hasattr(mylist, "__iter__"))  # True
print(hasattr(mylist, "__next__"))  # False
print(hasattr(it, "__iter__"))  # True
print(hasattr(it, "__next__"))  # True

The list is iterable, but not an iterator. iter(mylist) gives you the actual iterator.

Why Use Iterators?

Here are some characteristics that make iterators a great tool:

• Efficiency: When you use iterators such as open() to read files, they don’t store everything in memory.
• Laziness: They only compute when they’re called.
• Composability: Iterators integrate well into processing pipelines (like map, filter, etc.)

If you’re working with big data, reading files, streaming APIs, and so on, you can use iterators to process the data progressively.

The Iterator Protocol

If an object implements both __iter__() and __next__(), it’s an iterator. If you create a MyIterator class like the following, each instance of the class will be an iterator:

class CountDown:
    """A custom iterator that counts down from `start` to 0."""
    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self

    def __next__(self):
        if self.current < 0:
            raise StopIteration # iteration is over
        else:
        num = self.current
        self.current -= 1
        return num

In this example, the __iter__ method returns self because the object is its own iterator, this is what makes the instances of this class work inside loops. The __next__ manages the object's state using self.current (it returns the current value and decrements it for the next iteration.) This is a "lazy" process because it happens every time the object is called in a loop.

You can instantiate the above class in a loop, giving it an integer as an argument. For example, the following will output a count from 50 to 0:

for i in CountDown(50):
    print(i)

You can also use an object of the class in a while loop and use next() to get the next value, this gives you more control over the loop (you can decide when to exit, or if you want to wait within it for example):

countdown = CountDown(50)
while True:
    try:
        num = next(countdown)
        print(num)
    except StopIteration:
        print("Done!")
        break

Built-in Iterators

Several built-in Python functions return iterators directly, here’s a list:

• **zip()**: Joins items from multiple iterables
• **enumerate()**: Pairs items with their index
• **map()**: Applies a function lazily
• **filter()**: Filters items lazily
• **iter()**: Returns an iterator from an iterable
• **reversed()**: Iterator over reversed sequence
• **open()**: File object, yields lines

Let’s take zip() as an example, it allows to iterate over a group of iterables (such as lists), and return a tuple with the nth element of each iterable at each loop's iteration.

z = zip([1, 2], ['a', 'b'])
next(z) # (1, 'a')
next(z) # (2, 'b')

Unlike a list of tuples, zip() doesn’t build anything: it yields one pair at a time.

Fun Examples

Weird examples are always great for figuring out how things work. Let’s take a look at some!

Zipping fun

The zip() function is a reference to the zipper, it's an image of how 2 lists

names = ['Alice', 'Bob']
ages = [30, 25]
cities = ['Paris', 'Berlin']

for name, age, city in zip(names, ages, cities):
    print(f"{name} is {age} and lives in {city}")

You can also nest zips, it becomes unreadable quickly, but there you go

nums = [1, 2]
letters = ['a', 'b']
zipped = zip(nums, letters)
for pair in zip(zipped, [True, False]):
    print(pair)
# ((1, 'a'), True)
# ((2, 'b'), False)

Combining Iterators

You can also combine iterators, the following will twist your brain:

def add1(x):
    return x + 1

nums = [1, 2]
letters = ["a", "b"]
zipped = zip(nums, letters)
for index, pair in enumerate(zip(zipped, map(add1, reversed([2, 3])))):
    print(index, pair)
# 0 ((1, 'a'), 4) 
# 1 ((2, 'b'), 3)

In the example above:

• We loop using enumerate(), which yields the output of the index variable (0, then 1)
• Le loop traverses a zip of two iterables: zipped and the result of map(add1, reversed([2, 3]))
• zipped is also an iterator (a Zip object), yielding one tuple at a time
• map applies the add1 function (that increments a number by 1) to an iterable (reversed([2, 3]))
• reversed returns an iterator yielding the elements of a list ([2, 3]) in reversed order

Building Your Own Iterators

We’ve seen how to create an iterator class and how Python’s built-in iterators work. While you’ll often use existing iterators, there are cases where a custom iterator is the cleanest solution. Let’s explore a practical example.

Shuffer Cards

Imagine a game where players draw cards from a continuously reshuffled deck. When the deck is exhausted, it automatically reshuffles for the next round:

import random

class ShuffledDeck:
    def __init__(self):
        self.cards = list(range(1, 53))
        random.shuffle(self.cards)
        self.index = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self.index >= len(self.cards):
            print("new set of cards")
            random.shuffle(self.cards)
            self.index = 0
        card = self.cards[self.index]
        self.index += 1
        return card

This iterator never raises a StopIteration instead, when all the numbers in the list (that represent a card) are returned, it reshuffles the cards (random.shuffle(self.cards)) and it does for 52 more iterations.

Using a Generator Functions

Python has two main ways to create iterators: using classes (as we just saw), and using generator functions, which use the yield reserved word.

This is an example of a generator function:

def countdown(n):
    while n > 0:
        yield n
        n -= 1

Using yield turns the defined object (vountdown here) into a generator object. A generator object is an iterable; you can use it in a for loop (or use next):

for num in countdown(3):
    print(num)
# 3
# 2
# 1

Notice how you can use yield without returning (you can update the value after yielding it).

The yield keyword automatically makes the function into a generator function, which means if you use a return statement in it, it won't do anything (it won't raise an error either):

def weird(n):
    if n > 5:
        return n*2# this will never happen, because `yield` wins 
    else:
        while n <5:
        yield n
        n+=1
print(weird(6))
# <generator object weird at 0x78e2fdf12e00>

A Word About itertools

Python includes a standard library calleditertools, with tools for iterator manipulation. It has a whole bunch of them, for example: chain(), cycle(), count(), islice(), compress(), tee()...

These functions allow common iterations, so a first advantage of using this library is not having to define functions that already exist! For example, the count function... counts:

from itertools import count

for i in count(start=10):
    if i > 12:
        break
    print(i)
# 10
# 11
# 12

If you check the documentation, it states that this is equivalent to this Python implementation:

def count(firstval=0, step=1):  
    x = firstval  
    while 1:  
        yield x  
        x += step

But it’s only equivalent in terms of function output. Itertools is written in C, which makes the iterations faster than the Python implementation. While sometimes you need to sacrifice performance over readability (or vice-versa), this one is a no-brainer: Itertools will make your code cleaner and faster, so check the library and see if it solves your problem. If it doesn’t, you can always build your own Python iterator!

A Final Word

Python’s iterator protocol is quite a subject to learn! While looping is an intuitive process, understanding how it’s implemented in Python is a different thing! Whether you’re zipping lists, writing pipelines, or building tools, learning how things work makes you a better programmer. Hopefully, you also discovered new tools.

If you’ve made it this far, you can now spot an iterator, build one, and even create your own reusable data patterns. I hope this helped!

Thank you for reading!

If you like my content and want to connect:

👉 Check all my other accounts

☕ You can buy me a coffee

Thank you for reading!

If you like my content and want to connect:

👉 Check all my other accounts

☕ You can buy me a coffee

[embed]Python concepts Edit descriptionericnarro.medium.com

Thank you for being a part of the community

Before you go:

• Be sure to clap and follow the writer ️👏️️
• Follow us: **X | [LinkedIn](https://www.linkedin.com/company/inplainenglish/) | [YouTube](https://www.youtube.com/@InPlainEnglish) | [Newsletter](https://newsletter.plainenglish.io/) | [Podcast](https://open.spotify.com/show/7qxylRWKhvZwMz2WuEoua0) | [Twitch](https://twitch.tv/inplainenglish)**
• **Start your own free AI-powered blog on Differ** 🚀
• **Join our content creators community on Discord** 🧑🏻‍💻
• For more content, visit **plainenglish.io + [stackademic.com](https://stackademic.com/)**