Why Proper Types Matter and Duck Typing Not

by Szymon Lipiński
tags: python programming

Many Python programmers repeat that looking like a duck is much more important than if it’s really a duck. So, if you have an object passed to a function, then the class of the object is not important. The only important thing is if the passed object has a specific function, which you want to call.

If it walks like a duck and it quacks like a duck, then it must be a duck

https://en.wikipedia.org/wiki/Duck_typing

What if it’s not? In the Wikipedia there is a more historical term, a Duck Test:

If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck.

https://en.wikipedia.org/wiki/Duck_test

The word must is a very bad assumption. What if it’s not? Will the program blow up on production. The word probably is a better assumption, but it’s also a very significant source of problems in many programs I observed. Sometimes something looking like a duck is not a duck at all. It’s not even similar to a duck.

A string is not a number and a number is not a rectangle. So how come that in programming people claim these are the same things? Simple things will be simple. Unfortunately, every simple program tends to go in the direction of a much more twisted beast, right after you think about adding a new feature or two. Too often after such a change the result was having twisted ducks with four legs but without a head. The program was fine in most cases. Debugging was hard. It was much easier to start writing that from scratch. And then you will need to add a feature or two…

Should you care? Well, you always should. Unless you don’t care about the programs you are writing.

I Don’t Care What Type I Have

This is not true. You need to know if it’s a number, a string, or a dictionary. They have different interfaces, different set of functions, different fields etc. You use them differently.

Python’s love of Duck Typing brings us lots of limitations. Due to the lack of types in function declarations, you cannot have two functions with the same number of arguments and with different types. In C++ you can have:

int do_something(a: int);
int do_something(b: float);
int do_something(b: string);

Then the proper function to call will be chosen depending on the argument type you pass there.

In a duck typed language like Python you cannot have it. Instead you need to do:

def do_something(a):
	if isinstance(a, int):
		...
	if isinstance(a, float):
		...
	if isinstance(a, str):
		...

I’m sure it’s not simpler. I was criticized by a couple of programmers that it’s not the real Python way of writing programs. So what’s then if we need different behavior for different types? Just have three different functions with different names for each of the occasions? Something like do_something_int, do_something_float, do_something_string? Adding a type name into a function name is even worse when the refactoring time comes.

I Save So Much Typing

Some people claim that Duck Typing saves so much time as you don’t need to write all those terrible types. Well, it’s worse. Consider this simple C++ function:

/* Adds two numbers. */
int add(int a, int b) {
    return a + b;
}

It’s quite simple what it does, it takes two ints, returns two ints. I love the Haskell syntax for this:

add :: int -> int -> int

Now the Python version:

def add(a, b):
	return a + b

We have a couple of problems here:

Let’s add some documentation and let’s use type hints, which we can use in Python 3:

def add(a: int, b: int) -> int:
	"""Adds two numbers."""
	return a + b

So now the IDE can give you some amazingly useless hints. Useless because it assumes it will be int while it can also be MrDucky.

What about the saved key strokes?

C++ version: 67 characters
Python version: 76 characters

Function Is Good - It Has Hints

Check the previous function. It has hints. The sad reality is that you can still pass whatever you want and the function will happily work.

In [3]: add(1, 2)
Out[3]: 3

In [4]: add(1.0, 2.3)
Out[4]: 3.3

In [5]: add("1", "2")
Out[5]: '12'

It will work without error, but unfortunately not as you intended. It will return different types. You will need to take care of that later in your code, too often after getting some extremely critical production ticket.

The + operator for numbers and strings does something else, so the function returns something else. What’s more, the result is not what the function declares. It can be whatever. OK, one fix would be to convert the types to ints in the function like this:

def add(a: int, b: int) -> int:
	"""Adds two numbers."""
	return int(a) + int(b)

This way now we get an error and bad result for floats:

In [8]: add(1, 2)
Out[8]: 3

In [9]: add(1.7, 2.3)
Out[9]: 3

In [10]: add("1", "2")
Out[10]: 3

In [11]: add("1", "hey")
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-11-645ec93e6b54> in <module>()
----> 1 add("1", "hey")

<ipython-input-7-e0ec6e2a7a5c> in add(a, b)
      1 def add(a: int, b: int) -> int:
      2     """Adds two numbers."""
----> 3     return int(a) + int(b)

ValueError: invalid literal for int() with base 10: 'hey'

You can say: I don’t care, let the caller take care of passing the proper arguments. The problems is different. It’s the responsibility of the function author to make it work properly. However, caring about passing proper arguments means caring about the types, while we have Duck Typing, which promotes not caring about that.

You just have four ways to cope with this kinds of mess:

The C++ version will not compile if you pass anything else than integers, or a number type which can be casted to an integer. The caller needs to take care of the conversion. What does it give us? Well, we have much simpler model, it’s much easier to test the function if you know what types you get.

All the Python functions can be emulated in e.g. Java with functions which get objects of the Object class as the arguments. Or in C++ passing void* everywhere. Then to call an object’s method in the function, you need to cast and prey if the cast will be fine. If it’s not, then the program will crash or at least should crash (if you’re a fan of hiding casting errors).

Proper Operators Matter

You cannot say that you don’t care what is passed to a function. You always should care. This includes all kinds of logical errors where your objects have different behavior than the one you’d expect. This is the worst kind of errors you can get from a production system: no error in logs but something is terribly broken.

For instance in Haskell there is a different operator for adding numbers and a different for adding strings. It makes sense, as for strings it’s rather concatenation than adding. On the other hand having the same operator will not hurt, as long as you have strict type checking. Having a strict language with rules like: you must return what you declare, helps a lot, so you don’t return a type you didn’t want to.

Some comparisons are fine for some types, while some are not. Think about a simple integer. We can check if the value is smaller than some other value. We can also check the opposite way. We can also check if they are equal, which will mean that they are the same.

a = 10
b = 20
a < b
a > b
a == b

Of course all the rules you learned at school also matter here, so if you compare two numbers they can be either equal or the first can be smaller or bigger. There is no other possibility.

You can even define the operator == like:

if not a < b and not a > b

Let’s talk now about another type, which is a rectangle. This type will contain two fields: width and height. We can also calculate the area of a rectangle. We can do it only because we know that these two numbers represent width and height. If we would have a similar type with two numbers but with a meaning of height and age then calculating the area wouldn’t make any sense.

The integers in the previous example could be compared. But what would a comparison of rectangles mean?

a = Rectangle(1, 2)
b = Rectangle(3, 4)
a == b

This should be written as:

return a.width == b.width and a.height == b.height

What about smaller and bigger? When people say that one rectangle is bigger than another, they mean the area of this rectangle is bigger than the area of another. But comparing areas is just comparing numbers. So:

a < b means a.area < b.area
a > b means a.area > b.area

Notice that in the world of rectangles the rule from integers doesn’t apply, so you cannot replace == with:

if not a < b and not a > b

Take a look at the rectangles like these two:

a = Rectangle(3, 4)
b = Rectangle(6, 2)

The areas are the same, but the rectangles are not the same.

So the objects looks like a duck but are not even close to ducks and cannot be used one instead of another.

Impossible IDE Refactoring

IDEs are really helpful. They can automate many things. This includes refactoring like stripping out common code to a function or renaming a function used in many places. Take a look at this:

class A() {
    int do_something(int a, int b);
}
class B() {
    int do_something(int a, int b);
}

int x(A obj):
    return obj.do_something(1, 2);

int y(B obj):
    return obj.do_something(1, 2);

The refactoring I need to make is renaming the function do_something in the class A to something else. IDEs are great for this, just a couple of clicks. But to make it work the IDE needs to know where the function is used. If we have types, it’s pretty simple to see it.

Now the Python version:

class A() {
    def do_something(self, a, b)
}
class B() {
    def do_something(self, a, b)
}

def x(obj):
    return obj.do_something(1, 2);

def y(obj):
    return obj.do_something(1, 2);

Now the IDE has a problem. It cannot just do a simple string replacement as it will then replace also function in the class B. We can help a little bit with hints, but as you could see earlier, the hints are just hints, so I can pass anything I want.

This is not a problem with just the code as it is. Imagine a situation when you use an external library, it has hints, all works fine. Suddenly after an upgrade of the library, something starts to fail. The problem with this kind of hope-driven-programming is that usually the error will show you that an object twenty levels of function calls below doesn’t have an expected field. Happy debugging.

I’m criticized for my opinion with a simple: you should have tests for all that. Yep, you should. However writing tests to check the types is a huge overhead. Especially when I was told that I would be able to write less due to Duck Typing.

You should always have tests. I want to spent my time where it matters, so I want to write tests for the logic, for the corner cases, not to check if someone is passing proper types.

Remarks

At the first glance Duck Typing seems like a great idea. Especially for new programmers. For people who can just write something without knowing all the things below. They can just write something and it just works. It’s like driving a car without needing to know how it really works. If you really want to know how to program you need to know all the things below. Or at least as much as possible.

The Python projects I’ve been involved in had e.g. 160k lines of code (plus 50k of comments). Refactoring in such a huge project is terribly slow, even with tests. The whole process is: rename a function do_this in a class, use string search in IDE to search where the do_this string appears. Check all the places and wonder if this is the function I changed or maybe it’s a function from another class. Change a couple of function calls. Run tests. Check the failing ones, maybe they fail due to the last change or the lack of the change. Change the string in another place. Comparing this to refactoring in Kotlin or Java makes a huge difference. What’s more, after this refactoring the program just compiles. In Python I’m not even sure if it will run properly on production.