Generators in Python - Part 2

In the previous part of python generators, we understood what a generator is, how it can be used with for loops and how the generator method is different from a normal method. In this post, we will understand generator expression.

Why are generator expressions needed?

They help the process of the creation of generators and make it easy. A generator expression creates an anonymous generator function. Creating a generator expression is similar to creating a list comprehension in Python. The only difference between list comprehension and a generator expression is that a generator expression produces one data element at a time whereas a list comprehension produces the entire list.

Since the elements are produced only when they are needed, the amount of memory consumed when a generator expression is created is much less, in comparison to other options. This means generator expressions are memory-friendly.

Time for an example:

The below example shows the difference in output while using list comprehension and a generator expression.

my_list = [1, 12, 7, 89, 45]
print("The below is a list comprehension")
print([element**3 for element in my_list])  # list comprehension
print("\n")
print("The below is a generator expression")
(element**2 for element in my_list)  # generator expression

Output:

The below is a list comprehension
[1, 1728, 343, 704969, 91125]

The below is a generator expression
Out[6]: <generator object <genexpr> at 0x00000263B7991390>

The above code clearly shows that a list comprehension generated a list based on a specific operation, whereas a generator expression resulted in the creation of a generator object. This generator object returns data elements only when needed. This can be achieved with the help of the __next__ method.

Time for an example:

my_list = [1, 12, 7, 89, 45]
generator = (element**2 for element in my_list)  # generator expression
print(next(generator))
print(next(generator))
print(next(generator))
print(next(generator))
print(next(generator))
next(generator)

Output:

1
144
49
7921
2025
Traceback (most recent call last):

  File "<ipython-input-8-a66965a2b6ca>", line 8, in <module>
    next(generator)

StopIteration

---

Using generator expression with a built-in function

The generator expression can be used with built-in methods like sum, min, and max. Below is an example demonstrating the same:

my_list = [1, 12, 7, 89, 45]
generator_one = sum(element**2 for element in my_list)  # generator expression
generator_two = min(element**2 for element in my_list)  # generator expression
generator_three = max(element**2 for element in my_list)  # generator expression
print(generator_one)
print(generator_two)
print(generator_three)

Output:

10140
1
7921

---

Why use generators at all?

1. They are simple and easy to implement.

2. They keep all the bookkeeping automatically (creation of __iter__ and __next__ methods).

3. They are memory-friendly since the data element is generated on demand (one at a time), otherwise, only the generator object is the result.

4. They go hand-in-hand when dealing with large amounts of streaming data or bookkeeping data.

5. They are easier to understand and write as well!

---

Conclusion

In this post, we saw how generators are needed, how they can be used with built-in functions and their significance.