Python SWIG Bindings From C/C++

Overview

Sometimes you may want to call C++/C code from python. This could be because the functionality you need is contained in a C++ library, you require low-level system or I/O access, or you want the performance benefits of running C++/C code vs. python.

Although there are many different ways of calling C++ code from Python, this page focuses on using SWIG.

The SWIG logo.

The Interface File

SWIG is configured with what is called an interface file. The interface file tells SWIG where to find C/C++ files, what objects it should create wrappers for, and what features should be included.

Running SWIG

SWIG can be installed on Debian-like systems (e.g. Ubuntu) with the following command:

The basic invocation of SWIG for generated a python wrapper for C code is:

This will make SWIG ingest the interface file input.i and create a C wrapper file.

If you are creating a wrapper for C++ code instead of C, use:

SWIG Program Arguments:

Sorted alphabetically:

-includeall: SWIG parses #include directives from within source code, and generates bindings for all these files also. Usually not a good idea!

-threads: SWIG will release the GIL when calling any C/C++ library code from Python (excluding classes that have been wrapped in directors)

CMake

CMake has built-in support for creating SWIG bindings for your C++ project. To use CMake to build SWIG bindings, make sure to include the SWIG package with the following command in your CMakeLists.txt:

When using the CMake SWIG package, swig is automatically called for you by CMake. Don’t worry, you can still provide arguments to calling swig via the SET_SOURCE_FILES_PROPERTIES command.

You also need to provide a list of all the source files required to generated the library/bindings:

You create a Python module (a set of bindings created from settings in an interface file) by using the SWIG_ADD_MODULE() command.

 

Applying Compile Flags To The Generated SWIG Source File

SWIG generates a specific source file (normally with the extension .cxx if using C++) which contains the binding code which makes the C++ library compatible with Python.

If you want, you can modify the compiler flags used to compile this file with the ${swig_generated_file_fullname} parameter. For example, to compile with the C++11 standard:

${swig_generated_file_fullname} usually expands to <path>/<module name>PYTHON_wrap.cxx for C++ libraries. The variable is only valid after SWIG_ADD_MODULE(...) is called.

Installing Python Bindings Using CMake

You can instruct CMake to install the generated SWIG Python bindings onto your computer when sudo make install is run.

Add the following code to your CMakeLists.txt file, after the SWIG_ADD_MODULE(...) line.

Notice the mention of _<PythonModuleName>. The SWIG package automatically adds a underscore to the compiled C/C++ library, while the generated <PythonModuleName>.py file does not have this underscore.

 

Order Is Important

The order in which to add classes (or header files via %include) to the SWIG .i file is important. For example, is class B uses class A (say class B gets passed a pointer to a class A object in one of it’s methods), then class A should be included before class B, otherwise you won’t be able to pass in a python object of class A into B (it will complain about a TypeError).

Cross-language Polymorphism (Directors)

By default, cross-language polymorphism will not work with SWIG bindings. For example, consider a Python class which inherits from a wrapped C++ class containing virtual methods. If the child Python class then overrides some of the base class functions, and is passed into the C++ side, these overriding functions will not be called.

Cross-language polymorphism is important when you are using interfaces. Some C++ libraries use pure virtual (or just virtual) classes to describe interfaces to your application, which may be in Python.

Thankfully, SWIG supports cross language polymorphism. They support the feature through the use of directors. To enable directors, make sure directors="1" is appended to the module keyword as below:

However, you still need to tell SWIG what classes to create directors for. To do this, use %feature("director") as shown below. To enable directors for all classes which have virtual methods, add:

To enable directors for specific classes (recommended for larger projects as the code bloat that directors introduces can be quite high):

Threading Issues With Directors

Normally, you can pass in the -threads argument when calling SWIG to make SWIG release the GIL upon every entry to your C/C++ library from Python (or if you are using CMake, call SET_PROPERTY(SOURCE MyInterfaceFile.i PROPERTY SWIG_FLAGS "-threads") ). However, this is not the case for any C/C++ class that has been converted into a director.

I currently do not know how to make SWIG release the GIL when director methods are called from Python. This means that all other python threads will be blocked when C/C++ functions/classes that have been converted into directors are called from your Python code.

Output Variable Issues With Directors

Output variables passed into a C/C++ function (pointers or non-const references that are intended for relaying information back to the calling function, rather than using the return value) require careful treatment with SWIG. Even more so when the C/C++ functions are converted into directors.

The problem stems from the fact that Python has no concept of pointers. Which means that all parameters are passed by copy. This is o.k. for mutable Python objects, as it is always a reference to the object which is copied (i.e. similar behaviour to a pointer), but is a problem for basic immutable object types in Python such as integers or strings.

A C/C++ application which calls a base function which gets redirected to a Python child class (through the director mechanism) will not work correctly if it passes in pointers to basic data types such as integers and expects their value to be changed.

A work-around is to specify a SWIG typemap which converts the primitive C/C++ data type into a mutable Python object (such as a list with only one element, the primitive you are concerned about). Since a mutable Python object gets passed by taking a copy of the reference (works like a pointer to the object), any modifications in the callee code will be reflected in the caller’s code.

The disadvantage to this approach is that you have to remember that the variable passed into the Python code is now a list (or alternative mutable object), and you must use indexing to read/write the variable correctly.

We will work through an example that creates typemaps such that int * output variables will work correctly with directors.

Imagine we had this C++ code:

Place this code in your SWIG interface file (.i file):

Now we can do this in the Python code:

An alternative approach to converting the primitive C/C++ type into a mutable Python object is to use typemaps to extract the return value from a tuple that is returned from the called Python code. However, this can be messy as you have to map specific output variables to the returned tuple (but not all of them, only ones that are immutable types in Python).

What Did SWIG Create?

Sometimes, even though SWIG compiled successfully, it can be difficult to determine if SWIG create the python bindings as intended. There are some useful tools you can use within the Python environment to determine what SWIG has exactly created.

The first one worthy of mention is the Python help() command. Pass into help the module name of your bindings and Python will print all of the attributes of this module.

This allows you to see what functions, classes, enumerations e.t.c are available from the Python code.

Return Values

Whilst the C++ compiler enforces return values (if the function’s signature says that it will return a value), python does not. This can lead to issues when implementing concrete class in python of a base class that was defined in C++. If you forget to return the correct type, your program can segmentation fault.

Smart Pointers

As of September 2017, SWIG only has support for std::shared_ptr, but not std::unique_ptr or std::weak_ptr.

The SWIG support for  std::shared_ptr, is quite automatic. You have to first include the std::shared_ptr.i file in your SWIG interface file:

Once this is included, you then need to tell SWIG about all the types that will be used with a shared pointer. For example, if you want SWIG to handle std::shared_ptr<std::vector<uint8_t>> type variables, you would write:

Once this is done, shared pointers of this type should be created automatically when using this type with the SWIG interface.

However, I have run into a problem with this feature, in where I could no longer pass the underlying type back and forth if I had used the %shared_ptr() macro on it. So instead of enabling this special shared pointer type mapping, I created a custom type map for the shared pointer:

The above type map allows you to call C++ code that asks for a std::shared_ptr<std::vector<uint8_t>>  by passing in a standard Python list.

A similar type map was created for the other direction (C++ to Python) in the case that your are using directors:

This creates a Python list from a  std::shared_ptr<std::vector<uint8_t>>.

 

Posted: August 17th, 2017 at 4:35 am
Last Updated on: September 19th, 2017 at 11:25 am