typeconstraints: A Python function/method-argument and return-value type-constraint assertion decorator module.

Repository

https://github.com/pibara-utopian/typeconstraints

Introduction

If you have ever written code in a language like C++, you will appreciate the value of not being able to call a function with wrongly typed arguments or accidentally return a wrongly typed return value from a method. Trying to do so will in C++ lead to compile errors. The C++ compiler enforces type constraints. In Python we have no such safeguards. As a result, bugs can sneak into a Python program that would not have passed compilation had the program been written in C++. While we do have to work with runtime checks, even runtime we will want things to rather fail clearly and concisely at an as early as possible a point, than let code in some different part of the program, do something assuming a certain data structure, only to fail in some remote corner of the program with an often hard to track down symptom. The new typeconstraints decorator module for Python aims to alleviate the finding of these types of bugs by providing a simple way to put type constraints around functions and methods. Type constraints that while checked at runtime will expose type issues brought about through accidentally wrongly invoked functions or methods or accidental returning the wrong return type from such a method or function.

C++, vs python, an example

Let's look at a simple example function. The C++ example shows a whole lot of type info, We have a function taking two arguments, an integer and a matrix of dictionaries that all map strings to integers that returns a boolean.

bool transform_something(int x,
            std::vector<  std::vector< std::map< str::string, int >  > > &m) {
   ...
   return True
}

The Python equivalent would look simething like this:

def tranform_something(x,m):
    ...
    return True

Absolutely nothing in there telling you x is an integer or m is a matrix. If you accidentally pass a list of dicts as the second argument instead of a matrix of dicts, nothing in this part of the code will complain and if something goes wrong enough to throw an exception.

So how do we fix this? The Python language has the possibility to create decorators that allow you to use simple annotations to wrap functions and methods with additional functionality. You might know decorators for profiling for example. In this post I want to introduce the typeconstraints decorator and the helper classes that come with the typeconstraints module that make working with type constraints in python something relatively effortless.

pip3 install typeconstraints

Before we can use typeconstraints, the first we need to do is install the typeconstraints module on our system with pip3.
Now let us look at our example code to see what we can do with it.

from typeconstraints import typeconstraints, 
    NONNABLE,ANYOF, ARRAYOF,MIXEDDICT,MIXEDARAY

@typeconstraints([int,ARRAYOF(ARRAYOF(MIXEDDICT({"foo": int,"bar": int})))],
    [bool])
def tranform_something(x,m):
    ...
    return True

The above will provide you with type constraints nor unlike those imposed by the earlier C++ example. Not compile time unfortunately, runtime, runtime asserts. Now, these may be a few to many steps all at once let us look at a simpler example first.

from typeconstraints import typeconstraints

@typeconstraints([int, str][bool])
def foo(bar,baz):
   ..
   return True

ok = foo(17, "hi there")

This would be the run-time equivalent of something like:

bool foo(int bar, string baz) {
   ...
   return true;
}

auto ok = foo(17,"hi there");

The notational aspect is a bit different, but you see the Python and the C++ examples hold exactly the same type information. When you try to call the Python code with wrongly typed function arguments, the result will be an imediate AssertionError being thrown.

typeconstraints

The core of the typeconstraints module is the actual typeconstraints decorator. The decorator takes two arguments:

• A list of function argument typeconstraints
• A list of returnvalue typeconstraints.

A typeconstraint can be one of two things:

• An actual type
• A callable that takes exactly one argument (the function argument it is supposed to match)

The function argument typeconstraints are defined in the same order as the function arguments they refr to are in the function. The reason that the returntype is a list is that it is not uncommon in python to return a tuple instead of a single rval.

Callables as type constraints

As we saw above, type constraints can be types like int or str, but they can also be callables. In the next few sections we will describe a set of callables that the typeconstraints module comes with. Realize though that it is always possible to write your own custom typeconstraints. If you end up writing one, please consider sending me a pull request if you think other people might find your type constraint useful.

NONNABLE

The simplest callable that the typeconstraints module comes with is the NONNABLE.

from typeconstraints import typeconstraints, NONNABLE

@typeconstraints([float,NONNABLE(str)],[bool])
def do_something(num,msg=None):
    ...
    return True

ok1 = do_something(1.234,"ok")
ok2 = do_something(3.14159)

It is not uncommon that a function argument has a type but also can validly ne defined as None. For those cases the NONNABLE callable provides us with a usefull type constraint.

ANYOF

from typeconstraints import typeconstraints, ANYOF

@typeconstraints([ANYOF([int, float]),[bool])
def do_something_else(num):
    ...
    return True

ok1 = do_something_else(42)
ok2 = do_something_else(3.14159)

Not all functions function with just one type. For example, many functions may work with any type of number.
The ANYOF callable provides us with a means to describe an argument that can have one of a fixes set of types.

ARRAYOF

from typeconstraints import typeconstraints, ARAYOF

@typeconstraints([ARAYOF(float,minsize=1,maxsize=10)],[float])
def sum_floats(numbers):
    sum=0.0
    for num in numbers:
        sum += num
    return sum

val = sum_floats(3.1415,12.234,9.71)

Python lists are fexible. You can put strings, floats and even other lists as elements in the same list.
Often this is a bit too flexible and you really want to assert that a list of floats indeed is a list of floats. For that purpose, the ARAYOF callable provides a useful type constraint.

MIXEDDICT

from typeconstraints import typeconstraints, MIXEDDICT

@typeconstraints([MIXEDDICT({"foo": str, "bar": 17, "baz": false},
    optional=["baz"], ignore_extra=True)],[str])
def calculate_hash(objectdict):
    ...
    return sum

hash = calculate_hash({"foo": "Pieter", "bar": 77, "qux": False})

Wherein C++ a map maps between two types, a dict in Python will often be used as a multi-type record. The MIXEDDICT helps in cases where this is the reality. Every named key in the dict has its own typeconstraint defined for it.

AssertionError messages

So what when things go wrong. If a method gets called with wrong types for example. Well, in that case, an AssertionError is thrown. The messages coming from these exceptions can be rather verbose if the callables are used in a nested way. The important thing though is that they tell you exactly what went wrong, nested in the same way that the callable type constraints were. Here is an example of an exception message from a deeply nested set of callable type constraints:

AssertionError: 

foo25:Indexed argument 1 did not pass constraint function checking by <class 'typeconstraints.ARRAYOF'> 
[Argument of type <class 'list'>with length 2 and element number 1 of type <class 'list'> is not a valid 
ARRAYOF <typeconstraints.ARRAYOF object at 0x7f4c816b9ac8>because of invalid element type. 
[Argument of type <class 'list'>with length 1 and element number 0 of type <class 'dict'> is not a 
valid ARRAYOF <typeconstraints.MIXEDDICT object at 0x7f4c816b9a20>because of invalid element 
type. [Argument of type <class 'dict'> and named element 'baz' of type <class 'str'> not MIXEDDICT 
{'foo': <class 'str'>, 'bar': <typeconstraints.ANYOF object at 0x7f4c816b9b38>, 
'baz': <typeconstraints.NONNABLE object at 0x7f4c816b9a58>} 
[Argument of type <class 'str'> not NONNABLE type <class 'bool'> ]]]]

Conclusion

The Python typeconstraints module provides a simple but very powerful set of tools that could save you many hours of debugging by making your code fail early and with a verbose, yet clear, exception messages. If you ever suffered the types of bugs that lack of type constraints in languages like Python and JavaScript makes easy to make, you will surely appreciate the help from this little module in keeping you from spending precious time trying to track them down.