Which One Use: List Comprehension Or Map Part 2?
In the previous blog post I have presented results of benchmarking list comprehensions and maps. In this one I will run similar tests, but I will filter the list to use only the even numbers.
I will not repeat the test description from the previous article, you can find it there.
The Test Logic
The test logic is simple: sum all the squares of the even number from the range of (1..max). So basically:
data = list(range(1, MAX_NUMBER+1))
@profile
def sum_numbers(data):
res = 0
# different algorithms go here
return res
A couple of remarks:
The original list contains the same number of elements as in the previous post. I will just filter it to use only the even ones.
Filtering will be done using normal if
or if in list comprehension
or filter() when using map()
.
The filtering expression is: number % 2 == 0
.
For Python 2 I used: 2.7.12 (default, Jul 1 2016, 15:12:24) [GCC 5.4.0 20160609]
.
For Python 3 I used: 3.5.2 (default, Sep 10 2016, 08:21:44) [GCC 5.4.0 20160609]
.
I will not present the full results, instead there is just a table at the end.
A Simple For Loop
This is the simplest and the most common way of iterating a container. Just make a for loop.
@profile
def sum_numbers(data):
res = 0
for x in data:
if x % 2 == 0:
res += x*x
return res
A List Comprehension Squared With For Loop
Here we have a list comprehension, which creates a list of original values. The values are then squared and summed.
@profile
def sum_numbers_list_comprehension_for_square(data):
res = 0
for x in [n for n in data if n % 2 == 0]:
res += x*x
return res
A List Comprehension With For Loop Squaring
Here we have a list comprehension, which creates a list of squared values, and then it is summed using the for loop.
@profile
def sum_numbers_list_comprehension_for_square(data):
res = 0
for x in [n*n for n in data if n % 2 == 0]:
res += x
return res
A List Comprehension With Sum()
There is a list comprehension, which creates a list of squared values, and then it is summed using the sum
function.
@profile
def sum_numbers_list_comprehension_squared_sum(data):
res = 0
res = sum([n*n for n in data if n % 2 == 0])
return res
A Map With For Loop Squaring
Here we have the map()
function, which returns in fact an original values. They are then squared, and summed in a for loop.
@profile
def sum_numbers_map_for_square(data):
res = 0
for x in map(lambda n: n, filter(lambda m: m % 2 == 0, data)):
res += x*x
return res
A Map Squared With For Loop
Here the map
returns squared values, which are summed using the for loop.
@profile
def sum_numbers_map_squared_for(data):
res = 0
for x in map(lambda n: n*n, filter(lambda m: m % 2 == 0, data)):
res += x
return res
A Map Squared With Sum
And the most common functional way of implementing this: map
returning squared values, which are summed using the sum
function.
@profile
def sum_numbers_map_squared_sum(data):
res = 0
res = sum(map(lambda n: n*n, filter(lambda m: m % 2 == 0, data)))
return res
The Summary
Lot’s of data. Let’s sum it up.
The data from the previous post (so all functions without filtering):
Test Name | Python Version | Time [s] | Memory Jump [MB] |
---|---|---|---|
simple for loop | 2 | 68.41 | 0.0 |
simple for loop | 3 | 76.08 | 0.0 |
compr. for sq. | 2 | 105.65 | 9.7 |
compr. for sq. | 3 | 117.45 | 7.8 |
compr. sq. for | 2 | 105.33 | 9.2 |
compr. sq. for | 3 | 117.23 | 7.8 |
compr. sum | 2 | 34.68 | 41.6 |
compr. sum | 3 | 37.74 | 38.8 |
map for. sq. | 2 | 139.67 | 7.6 |
map for. sq. | 3 | 150.95 | 0.0 |
map sq. for | 2 | 141.77 | 31.1 |
map sq. for | 3 | 159.23 | 0.0 |
map sq. sum | 2 | 71.23 | 31.1 |
map sq. sum | 3 | 77.07 | 0.0 |
This time data:
Test Name | Python Version | Time [s] | Memory Jump [MB] |
---|---|---|---|
simple for loop | 2 | 86.43 | 0.0 |
simple for loop | 3 | 95.12 | 0.0 |
compr. for sq. | 2 | 70.00 | 3.7 |
compr. for sq. | 3 | 76.40 | 3.9 |
compr. sq. for | 2 | 70.21 | 17.0 |
compr. sq. for | 3 | 77.28 | 19.5 |
compr. sum | 2 | 35.70 | 17.0 |
compr. sum | 3 | 38.39 | 19.4 |
map for. sq. | 2 | 146.33 | 11.5 |
map for. sq. | 3 | 155.91 | 0.0 |
map sq. for | 2 | 138.70 | 23.3 |
map sq. for | 3 | 156.66 | 0.0 |
map sq. sum | 2 | 109.05 | 8.6 |
map sq. sum | 3 | 119.87 | 0.0 |
I’m interested only in the memory jump inside the test function. I ignore it if the memory decreased later.
A couple of remarks:
- The simple for loop is not the best.
- The memory overhead is much smaller than without filtering, which is not a surprise, as we need to store half of the data.
- Map is still much slower than list comprehension.
- Map on Python 3 has almost no overhead - which is obvious when you read the documentation and find out that the
filter
and themap
are generators, so they return one value at a time instead of building the whole list. - The list comprehension has memory overhead, as it builds a new list, so generally it should be slower, and has bigger memory usage.
- Python 3 is slower than Python 2.
The Final Remark
After all the benchmarks I found out one thing.
That’s not true that
You should use list comprehension because that’s more pythonic.
The truth is
You should use list comprehensions, not map, because they are MUCH FASTER
Howgh