In a couple of previous posts I was considering which is better in Python: map or list comprehension. The final idea was that none of them is as fast as a C++ program, even using exactly the same algorithm.
What about changing an algorithm a little bit?
In the previous blog post I have presented results of benchmarking list comprehensions and maps in Python.
The results looked fine, however there was one thing I couldn’t stop thinking about. The huge memory overhead and the huge execution time.
The process used 50MB of memory from the beginning. Then it had just to iterate through a list of integers (which are small). The number of integers is also small (only 1M). Then it had to make 1M divisions to find the even numbers, and square it (making 0.5M multiplications, and 0.5M additions).
Why did it take 1.5 minute?
In the previous blog post I have presented results of benchmarking list comprehensions and maps. In this one I will run similar tests, but I will filter the list to use only the even numbers.
I will not repeat the test description from the previous article, you can find it there.
Some time ago there was a discussion on IRC, which looked like this:
- You should not use the map() function
in Python but the only list comprehension.
- Why?
- Because it is more Pythonic.
- Why?
- Because all people use the list comprehension.
Other arguments were not convincing as well. I have written about the map and the list comprehension in Advanced Python Constructs.
Arguments like “because I say it” or “all people do it like this” (evidence please) are just stupid, not convincing, and simply false (I use map so that’s not true that all people don't use it). So I have decided to make some benchmarks for the memory/time characteristics for list comprehensions and maps.
In a Django project I’m working on, I have a couple of similar models. I will show how to refactor the models to have less code, clearer project, and remove lots of repeated code.
Let’s assume that I’m working on indexing youtube videos.