Rust bindings for Python. This includes running and interacting with Python code from a Rust binary, as well as writing native Python modules.

• User Guide: stable | master

• API Documentation: master

A comparison with rust-cpython can be found in the guide.

PyO3 supports Python 3.5 and up. The minimum required Rust version is 1.37.0-nightly 2019-07-19.

If you have never used nightly Rust, the official guide has a great section about installing it.

PyPy is also supported (via cpyext) for Python 3.5 only, targeted PyPy version is 7.0.0. Please refer to the pypy section.

You can either write a native Python module in Rust, or use Python from a Rust binary.

However, on some OSs, you need some additional packages. E.g. if you are on Ubuntu 18.04, please run

sudo apt install python3-dev python-dev

PyO3 can be used to generate a native Python module.

Cargo.toml

[package]
name = "string-sum"
version = "0.1.0"
edition = "2018"

[lib]
name = "string_sum"
crate-type = ["cdylib"]

[dependencies.pyo3]
version = "0.8.5"
features = ["extension-module"]

src/lib.rs

# #![allow(unused_variables)]
#fn main() {
use pyo3::prelude::*;
use pyo3::wrap_pyfunction;

#[pyfunction]
/// Formats the sum of two numbers as string
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
    Ok((a + b).to_string())
}

/// This module is a python module implemented in Rust.
#[pymodule]
fn string_sum(py: Python, m: &PyModule) -> PyResult<()> {
    m.add_wrapped(wrap_pyfunction!(sum_as_string))?;

    Ok(())
}
#}

On Windows and Linux, you can build normally with cargo build --release. On macOS, you need to set additional linker arguments. One option is to compile with cargo rustc --release -- -C link-arg=-undefined -C link-arg=dynamic_lookup, the other is to create a .cargo/config with the following content:

[target.x86_64-apple-darwin]
rustflags = [
  "-C", "link-arg=-undefined",
  "-C", "link-arg=dynamic_lookup",
]

For developing, you can copy and rename the shared library from the target folder: On MacOS, rename libstring_sum.dylib to string_sum.so, on Windows libstring_sum.dll to string_sum.pyd and on Linux libstring_sum.so to string_sum.so. Then open a Python shell in the same folder and you'll be able to import string_sum.

To build, test and publish your crate as a Python module, you can use maturin or setuptools-rust. You can find an example for setuptools-rust in examples/word-count, while maturin should work on your crate without any configuration.

Add pyo3 to your Cargo.toml like this:

[dependencies]
pyo3 = "0.8.5"

Example program displaying the value of sys.version and the current user name:

use pyo3::prelude::*;
use pyo3::types::IntoPyDict;

fn main() -> Result<(), ()> {
    let gil = Python::acquire_gil();
    let py = gil.python();
    main_(py).map_err(|e| {
        // We can't display python error type via ::std::fmt::Display,
        // so print error here manually.
        e.print_and_set_sys_last_vars(py);
    })
}

fn main_(py: Python) -> PyResult<()> {
    let sys = py.import("sys")?;
    let version: String = sys.get("version")?.extract()?;
    let locals = [("os", py.import("os")?)].into_py_dict(py);
    let code = "os.getenv('USER') or os.getenv('USERNAME') or 'Unknown'";
    let user: String = py.eval(code, None, Some(&locals))?.extract()?;
    println!("Hello {}, I'm Python {}", user, version);
    Ok(())
}

Our guide has a section with lots of examples about this topic.

• examples/word-count Counting the occurrences of a word in a text file
• hyperjson A hyper-fast Python module for reading/writing JSON data using Rust's serde-json
• rust-numpy Rust binding of NumPy C-API
• html-py-ever Using html5ever through kuchiki to speed up html parsing and css-selecting.
• pyo3-built Simple macro to expose metadata obtained with the built crate as a PyDict
• point-process High level API for pointprocesses as a Python library
• autopy A simple, cross-platform GUI automation library for Python and Rust.
  • Contains an example of building wheels on TravisCI and appveyor using cibuildwheel
• orjson Fast Python JSON library
• inline-python Inline Python code directly in your Rust code
• Rogue-Gym Customizable rogue-like game for AI experiments
  • Contains an example of building wheels on Azure Pipelines
• fastuuid Python bindings to Rust's UUID library
• python-ext-wasm Python library to run WebAssembly binaries
• dict-derive Derive FromPyObject to automatically transform Python dicts into Rust structs

PyO3 provides some handy traits to convert between Python types and Rust types.

The easiest way to convert a Python object to a Rust value is using .extract()?.

ToPyObject trait is a conversion trait that allows various objects to be converted into PyObject. IntoPy<PyObject> serves the same purpose, except that it consumes self.

There are several ways how to pass positional and keyword arguments to a Python object call. The ObjectProtocol trait provides two methods:

• call - call any callable Python object.
• call_method - call a specific method on the object, shorthand for get_attr then call.

Both methods accept args and kwargs arguments.

use pyo3::prelude::*;
use pyo3::types::{PyDict, PyTuple};

struct SomeObject;
impl SomeObject {
    fn new(py: Python) -> PyObject {
        PyDict::new(py).to_object(py)
    }
}

fn main() {
    let arg1 = "arg1";
    let arg2 = "arg2";
    let arg3 = "arg3";

    let gil = Python::acquire_gil();
    let py = gil.python();

    let obj = SomeObject::new(py);

    // call object without empty arguments
    obj.call0(py);

    // call object with PyTuple
    let args = PyTuple::new(py, &[arg1, arg2, arg3]);
    obj.call1(py, args);

    // pass arguments as rust tuple
    let args = (arg1, arg2, arg3);
    obj.call1(py, args);
}

kwargs can be None or Some(&PyDict). You can use the IntoPyDict trait to convert other dict-like containers, e.g. HashMap, BTreeMap as well as tuples with up to 10 elements and Vecs where each element is a two-element tuple.

use pyo3::prelude::*;
use pyo3::types::{IntoPyDict, PyDict};
use std::collections::HashMap;

struct SomeObject;

impl SomeObject {
    fn new(py: Python) -> PyObject {
        PyDict::new(py).to_object(py)
    }
}

fn main() {
    let key1 = "key1";
    let val1 = 1;
    let key2 = "key2";
    let val2 = 2;

    let gil = Python::acquire_gil();
    let py = gil.python();

    let obj = SomeObject::new(py);

    // call object with PyDict
    let kwargs = [(key1, val1)].into_py_dict(py);
    obj.call(py, (), Some(kwargs));

    // pass arguments as Vec
    let kwargs = vec![(key1, val1), (key2, val2)];
    obj.call(py, (), Some(kwargs.into_py_dict(py)));

    // pass arguments as HashMap
    let mut kwargs = HashMap::<&str, i32>::new();
    kwargs.insert(key1, 1);
    obj.call(py, (), Some(kwargs.into_py_dict(py)));
}

Many conversions in PyO3 can't use std::convert::Into because they need a GIL token. That's why the IntoPy<T> trait offers an into_py method that works just like into, except for taking a Python<'_> argument.

Eventually, traits such as ToPyObject will be replaced by this trait and a FromPy trait will be added that will implement IntoPy, just like with From and Into.

You can use the create_exception! macro to define a new exception type:

# #![allow(unused_variables)]
#fn main() {
use pyo3::create_exception;

create_exception!(module, MyError, pyo3::exceptions::Exception);
#}

• module is the name of the containing module.
• MyError is the name of the new exception type.

For example:

use pyo3::prelude::*;
use pyo3::create_exception;
use pyo3::types::IntoPyDict;
use pyo3::exceptions::Exception;

create_exception!(mymodule, CustomError, Exception);

fn main() {
    let gil = Python::acquire_gil();
    let py = gil.python();
    let ctx = [("CustomError", py.get_type::<CustomError>())].into_py_dict(py);

    py.run("assert str(CustomError) == \"<class 'mymodule.CustomError'>\"", None, Some(&ctx)).unwrap();
    py.run("assert CustomError('oops').args == ('oops',)", None, Some(&ctx)).unwrap();
}

To raise an exception, first you need to obtain an exception type and construct a new PyErr, then call the PyErr::restore() method to write the exception back to the Python interpreter's global state.

use pyo3::{Python, PyErr};
use pyo3::exceptions;

fn main() {
    let gil = Python::acquire_gil();
    let py = gil.python();
    PyErr::new::<exceptions::TypeError, _>("Error").restore(py);
    assert!(PyErr::occurred(py));
    drop(PyErr::fetch(py));
}

If you already have a Python exception instance, you can simply call PyErr::from_instance().

PyErr::from_instance(py, err).restore(py);

If a Rust type exists for the exception, then it is possible to use the new method. For example, each standard exception defined in the pyo3::exceptions module has a corresponding Rust type, exceptions defined by create_exception! and import_exception! macro have Rust types as well.

# #![allow(unused_variables)]
#fn main() {
# use pyo3::exceptions;
# use pyo3::prelude::*;
# fn check_for_error() -> bool {false}
fn my_func(arg: PyObject) -> PyResult<()> {
    if check_for_error() {
        Err(exceptions::ValueError::py_err("argument is wrong"))
    } else {
        Ok(())
    }
}
#}

Python has an isinstance method to check an object's type, in PyO3 there is a Python::is_instance() method which does the same thing.

use pyo3::Python;
use pyo3::types::{PyBool, PyList};

fn main() {
    let gil = Python::acquire_gil();
    let py = gil.python();
    assert!(py.is_instance::<PyBool, _>(PyBool::new(py, true)).unwrap());
    let list = PyList::new(py, &[1, 2, 3, 4]);
    assert!(!py.is_instance::<PyBool, _>(list.as_ref()).unwrap());
    assert!(py.is_instance::<PyList, _>(list.as_ref()).unwrap());
}

Python::is_instance() calls the underlying PyType::is_instance method to do the actual work.

To check the type of an exception, you can simply do:

# use pyo3::exceptions;
# use pyo3::prelude::*;
# fn main() {
# let gil = Python::acquire_gil();
# let py = gil.python();
# let err = exceptions::TypeError::py_err(());
err.is_instance::<exceptions::TypeError>(py);
# }

The vast majority of operations in this library will return PyResult<T>. This is an alias for the type Result<T, PyErr>.

A PyErr represents a Python exception. Errors within the PyO3 library are also exposed as Python exceptions.

The PyO3 library handles Python exceptions in two stages. During the first stage, a PyErr instance is created. At this stage, holding Python's GIL is not required. During the second stage, an actual Python exception instance is created and set active in the Python interpreter.

In simple cases, for custom errors adding an implementation of std::convert::From<T> trait for this custom error is enough. PyErr::new accepts an argument in the form of ToPyObject + 'static. If the 'static constraint can not be satisfied or more complex arguments are required, the PyErrArguments trait can be implemented. In that case, actual exception argument creation is delayed until a Python object is available.

# #![allow(unused_variables)]
#fn main() {
# use pyo3::{exceptions, PyErr, PyResult};
# use std::error::Error;
# use std::fmt;
#
# #[derive(Debug)]
# struct CustomIOError;
#
# impl Error for CustomIOError {}
#
# impl fmt::Display for CustomIOError {
#     fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
#         write!(f, "Oh no!")
#     }
# }
#
# fn bind(_addr: &str) -> Result<(), CustomIOError> {
#     Err(CustomIOError)
# }

impl std::convert::From<CustomIOError> for PyErr {
    fn from(err: CustomIOError) -> PyErr {
        exceptions::OSError::py_err(err.to_string())
    }
}

fn connect(s: String) -> PyResult<bool> {
    bind("127.0.0.1:80")?;
    Ok(true)
}
#}

The code snippet above will raise an OSError in Python if bind() returns a CustomIOError.

The std::convert::From<T> trait is implemented for most of the Rust standard library's error types so the try! macro or the ? operator can be used.

# #![allow(unused_variables)]
#fn main() {
use pyo3::prelude::*;

fn parse_int(s: String) -> PyResult<usize> {
    Ok(s.parse::<usize>()?)
}
#}

The code snippet above will raise a ValueError in Python if String::parse() returns an error.

It is possible to use an exception defined in Python code as a native Rust type. The import_exception! macro allows importing a specific exception class and defines a zero-sized Rust type for that exception.

# #![allow(unused_variables)]
#fn main() {
use pyo3::prelude::*;
use pyo3::import_exception;

import_exception!(io, UnsupportedOperation);

fn tell(file: PyObject) -> PyResult<u64> {
    use pyo3::exceptions::*;

    let gil = Python::acquire_gil();
    let py = gil.python();

    match file.call_method0(py, "tell") {
        Err(_) => Err(UnsupportedOperation::py_err("not supported: tell")),
        Ok(x) => x.extract::<u64>(py),
    }
}

#}

pyo3::exceptions defines exceptions for several standard library modules.

As shown in the Getting Started chapter, you can create a module as follows:

use pyo3::prelude::*;

// add bindings to the generated python module
// N.B: names: "librust2py" must be the name of the `.so` or `.pyd` file
/// This module is implemented in Rust.
#[pymodule]
fn rust2py(py: Python, m: &PyModule) -> PyResult<()> {

    // PyO3 aware function. All of our python interfaces could be declared in a separate module.
    // Note that the `#[pyfn()]` annotation automatically converts the arguments from
    // Python objects to Rust values; and the Rust return value back into a Python object.
    #[pyfn(m, "sum_as_string")]
    fn sum_as_string_py(_py: Python, a:i64, b:i64) -> PyResult<String> {
       let out = sum_as_string(a, b);
       Ok(out)
    }

    Ok(())
}

// logic implemented as a normal rust function
fn sum_as_string(a:i64, b:i64) -> String {
    format!("{}", a + b).to_string()
}

# fn main() {}

The #[pymodule] procedural macro attribute takes care of exporting the initialization function of your module to Python. It can take as an argument the name of your module, which must be the name of the .so or .pyd file; the default is the Rust function's name.

To import the module, either copy the shared library as described in Get Started or use a tool, e.g. maturin develop with maturin or python setup.py develop with setuptools-rust.

The Rust doc comments of the module initialization function will be applied automatically as the Python doc string of your module.

import rust2py

print(rust2py.__doc__)

Which means that the above Python code will print This module is implemented in Rust..

In Python, modules are first class objects. This means that you can store them as values or add them to dicts or other modules:

# #![allow(unused_variables)]
#fn main() {
use pyo3::prelude::*;
use pyo3::{wrap_pyfunction, wrap_pymodule};
use pyo3::types::IntoPyDict;

#[pyfunction]
fn subfunction() -> String {
    "Subfunction".to_string()
}

#[pymodule]
fn submodule(_py: Python, module: &PyModule) -> PyResult<()> {
    module.add_wrapped(wrap_pyfunction!(subfunction))?;
    Ok(())
}

#[pymodule]
fn supermodule(_py: Python, module: &PyModule) -> PyResult<()> {
    module.add_wrapped(wrap_pymodule!(submodule))?;
    Ok(())
}

fn nested_call() {
    let gil = GILGuard::acquire();
    let py = gil.python();
    let supermodule = wrap_pymodule!(supermodule)(py);
    let ctx = [("supermodule", supermodule)].into_py_dict(py);

    py.run("assert supermodule.submodule.subfunction() == 'Subfunction'", None, Some(&ctx)).unwrap();
}
#}

This way, you can create a module hierarchy within a single extension module.

PyO3 supports two ways to define a free function in Python. Both require registering the function to a module.

One way is defining the function in the module definition.

use pyo3::prelude::*;

#[pymodule]
fn rust2py(py: Python, m: &PyModule) -> PyResult<()> {

    // Note that the `#[pyfn()]` annotation automatically converts the arguments from
    // Python objects to Rust values; and the Rust return value back into a Python object.
    #[pyfn(m, "sum_as_string")]
    fn sum_as_string_py(_py: Python, a:i64, b:i64) -> PyResult<String> {
       Ok(format!("{}", a + b))
    }

    Ok(())
}

# fn main() {}

The other is annotating a function with #[pyfunction] and then adding it to the module using the wrap_pyfunction! macro.

use pyo3::prelude::*;
use pyo3::wrap_pyfunction;

#[pyfunction]
fn double(x: usize) -> usize {
    x * 2
}

#[pymodule]
fn module_with_functions(py: Python, m: &PyModule) -> PyResult<()> {
    m.add_wrapped(wrap_pyfunction!(double)).unwrap();

    Ok(())
}

# fn main() {}

Both the #[pyfunction] and #[pyfn] attributes support specifying details of argument parsing. The details are given in the section "Method arguments" in the Classes chapter. Here is an example for a function that accepts arbitrary keyword arguments (**kwargs in Python syntax) and returns the number that was passed:

# extern crate pyo3;
use pyo3::prelude::*;
use pyo3::wrap_pyfunction;
use pyo3::types::PyDict;

#[pyfunction(kwds="**")]
fn num_kwds(kwds: Option<&PyDict>) -> usize {
    kwds.map_or(0, |dict| dict.len())
}

#[pymodule]
fn module_with_functions(py: Python, m: &PyModule) -> PyResult<()> {
    m.add_wrapped(wrap_pyfunction!(num_kwds)).unwrap();
    Ok(())
}

# fn main() {}

In order to make the function signature available to Python to be retrieved via inspect.signature, use the #[text_signature] annotation as in the example below. The / signifies the end of positional-only arguments.

# #![allow(unused_variables)]
#fn main() {
use pyo3::prelude::*;

/// This function adds two unsigned 64-bit integers.
#[pyfunction]
#[text_signature = "(a, b, /)"]
fn add(a: u64, b: u64) -> u64 {
    a + b
}
#}

This also works for classes and methods:

# #![allow(unused_variables)]
#fn main() {
use pyo3::prelude::*;
use pyo3::types::PyType;

// it works even if the item is not documented:

#[pyclass]
#[text_signature = "(c, d, /)"]
struct MyClass {}

#[pymethods]
impl MyClass {
    // the signature for the constructor is attached
    // to the struct definition instead.
    #[new]
    fn new(obj: &PyRawObject, c: i32, d: &str) {
        obj.init(Self {});
    }
    // the self argument should be written $self
    #[text_signature = "($self, e, f)"]
    fn my_method(&self, e: i32, f: i32) -> i32 {
        e + f
    }
    #[classmethod]
    #[text_signature = "(cls, e, f)"]
    fn my_class_method(cls: &PyType, e: i32, f: i32) -> i32 {
        e + f
    }
    #[staticmethod]
    #[text_signature = "(e, f)"]
    fn my_static_method(e: i32, f: i32) -> i32 {
        e + f
    }
}
#}

Alternatively, simply make sure the first line of your docstring is formatted like in the following example. Please note that the newline after the -- is mandatory. The / signifies the end of positional-only arguments. This is not a feature of this library in particular, but the general format used by CPython for annotating signatures of built-in functions.

#[text_signature] should be preferred, since it will override automatically generated signatures when those are added in a future version of PyO3.

# #![allow(unused_variables)]
#fn main() {
use pyo3::prelude::*;

/// add(a, b, /)
/// --
///
/// This function adds two unsigned 64-bit integers.
#[pyfunction]
fn add(a: u64, b: u64) -> u64 {
    a + b
}

// a function with a signature but without docs. Both blank lines after the `--` are mandatory.

/// sub(a, b, /)
/// --
///
///
#[pyfunction]
fn sub(a: u64, b: u64) -> u64 {
    a - b
}
#}

When annotated like this, signatures are also correctly displayed in IPython.

>>> pyo3_test.add?
Signature: pyo3_test.add(a, b, /)
Docstring: This function adds two unsigned 64-bit integers.
Type:      builtin_function_or_method

Currently, there are no conversions between Fns in Rust and callables in Python. This would definitely be possible and very useful, so contributions are welcome. In the meantime, you can do the following:

You can use ObjectProtocol::is_callable to check if you got a callable, which is true for functions (including lambdas), methods and objects with a __call__ method. You can call the object with ObjectProtocol::call with the args as first parameter and the kwargs (or None) as second parameter. There are also ObjectProtocol::call0 with no args and ObjectProtocol::call1 with only the positional args.

If you have a static function, you can expose it with #[pyfunction] and use wrap_pyfunction! to get the corresponding PyObject. For dynamic functions, e.g. lambda and functions that were passed as arguments, you must put them in some kind of owned container, e.g. a box. (A long-term solution will be a special container similar to wasm-bindgen's Closure). You can then use a #[pyclass] struct with that container as a field as a way to pass the function over the FFI barrier. You can even make that class callable with __call__ so it looks like a function in Python code.

To define a custom Python class, a Rust struct needs to be annotated with the #[pyclass] attribute.

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;

#[pyclass]
struct MyClass {
   num: i32,
   debug: bool,
}
#}

The above example generates implementations for PyTypeInfo and PyTypeObject for MyClass.

You can use pyclasses like normal rust structs.

However, if instantiated normally, you can't treat pyclasses as Python objects.

To get a Python object which includes pyclass, we have to use some special methods.

PyRef is a special reference, which ensures that the referred struct is a part of a Python object, and you are also holding the GIL.

You can get an instance of PyRef by PyRef::new, which does 3 things:

1. Allocates a Python object in the Python heap
2. Copies the Rust struct into the Python object
3. Returns a reference to it

You can use PyRef just like &T, because it implements Deref<Target=T>.

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
# use pyo3::types::PyDict;
#[pyclass]
struct MyClass {
   num: i32,
   debug: bool,
}
let gil = Python::acquire_gil();
let py = gil.python();
let obj = PyRef::new(py, MyClass { num: 3, debug: true }).unwrap();
assert_eq!(obj.num, 3);
let dict = PyDict::new(py);
// You can treat a `PyRef` as a Python object
dict.set_item("obj", obj).unwrap();
#}

PyRefMut is a mutable version of PyRef.

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
#[pyclass]
struct MyClass {
   num: i32,
   debug: bool,
}
let gil = Python::acquire_gil();
let py = gil.python();
let mut obj = PyRefMut::new(py, MyClass { num: 3, debug: true }).unwrap();
obj.num = 5;
#}

Py is an object wrapper which stores an object longer than the GIL lifetime.

You can use it to avoid lifetime problems.

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
#[pyclass]
struct MyClass {
   num: i32,
}
fn return_myclass() -> Py<MyClass> {
    let gil = Python::acquire_gil();
    let py = gil.python();
    Py::new(py, MyClass { num: 1 }).unwrap()
}
let gil = Python::acquire_gil();
let obj = return_myclass();
assert_eq!(obj.as_ref(gil.python()).num, 1);
#}

The #[pyclass] macro accepts the following parameters:

• name=XXX - Set the class name shown in Python code. By default, the struct name is used as the class name.
• freelist=XXX - The freelist parameter adds support of free allocation list to custom class. The performance improvement applies to types that are often created and deleted in a row, so that they can benefit from a freelist. XXX is a number of items for the free list.
• gc - Classes with the gc parameter participate in Python garbage collection. If a custom class contains references to other Python objects that can be collected, the PyGCProtocol trait has to be implemented.
• weakref - Adds support for Python weak references.
• extends=BaseType - Use a custom base class. The base BaseType must implement PyTypeInfo.
• dict - Adds __dict__ support, so that the instances of this type have a dictionary containing arbitrary instance variables.
• module="XXX" - Set the name of the module the class will be shown as defined in. If not given, the class will be a virtual member of the builtins module.
• subclass - Allows Python classes to inherit from this class. This feature is hidden behind a unsound-subclass feature because it is currently causing segmentation faults

By default it is not possible to create an instance of a custom class from Python code. To declare a constructor, you need to define a method and annotate it with the #[new] attribute. Only Python's __new__ method can be specified, __init__ is not available.

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
# use pyo3::PyRawObject;
#[pyclass]
struct MyClass {
   num: i32,
}

#[pymethods]
impl MyClass {

     #[new]
     fn new(obj: &PyRawObject, num: i32) {
         obj.init({
             MyClass {
                 num,
             }
         });
     }
}
#}

Rules for the new method:

• If no method marked with #[new] is declared, object instances can only be created from Rust, but not from Python.
• The first parameter is the raw object and the custom new method must initialize the object with an instance of the struct using the init method. The type of the object may be the type object of a derived class declared in Python.
• The first parameter must have type &PyRawObject.
• For details on the parameter list, see the Method arguments section below.
• The return value must be T or PyResult<T> where T is ignored, so it can be just () as in the example above.

By default, PyObject is used as the base class. To override this default, use the extends parameter for pyclass with the full path to the base class. The new method of subclasses must call their parent's new method.

# use pyo3::prelude::*;
# use pyo3::PyRawObject;
#[pyclass]
struct BaseClass {
   val1: usize,
}

#[pymethods]
impl BaseClass {
   #[new]
   fn new(obj: &PyRawObject) {
       obj.init(BaseClass { val1: 10 });
   }

   pub fn method(&self) -> PyResult<()> {
      Ok(())
   }
}

#[pyclass(extends=BaseClass)]
struct SubClass {
   val2: usize,
}

#[pymethods]
impl SubClass {
   #[new]
   fn new(obj: &PyRawObject) {
       obj.init(SubClass { val2: 10 });
       BaseClass::new(obj);
   }

   fn method2(&self) -> PyResult<()> {
      self.get_base().method()
   }
}

The ObjectProtocol trait provides a get_base() method, which returns a reference to the instance of the base struct.

Property descriptor methods can be defined in a #[pymethods] impl block only and have to be annotated with #[getter] and #[setter] attributes. For example:

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
# #[pyclass]
# struct MyClass {
#    num: i32,
# }
#
#[pymethods]
impl MyClass {

     #[getter]
     fn num(&self) -> PyResult<i32> {
        Ok(self.num)
     }
}
#}

A getter or setter's function name is used as the property name by default. There are several ways how to override the name.

If a function name starts with get_ or set_ for getter or setter respectively, the descriptor name becomes the function name with this prefix removed. This is also useful in case of Rust keywords like type (raw identifiers can be used since Rust 2018).

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
# #[pyclass]
# struct MyClass {
#    num: i32,
# }
#
#[pymethods]
impl MyClass {

     #[getter]
     fn get_num(&self) -> PyResult<i32> {
        Ok(self.num)
     }

     #[setter]
     fn set_num(&mut self, value: i32) -> PyResult<()> {
        self.num = value;
        Ok(())
     }
}
#}

In this case, a property num is defined and available from Python code as self.num.

Both the #[getter] and #[setter] attributes accept one parameter. If this parameter is specified, it is used as the property name, i.e.

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
# #[pyclass]
# struct MyClass {
#    num: i32,
# }
#
#[pymethods]
impl MyClass {

     #[getter(number)]
     fn num(&self) -> PyResult<i32> {
        Ok(self.num)
     }

     #[setter(number)]
     fn set_num(&mut self, value: i32) -> PyResult<()> {
        self.num = value;
        Ok(())
     }
}
#}

In this case, the property number is defined and available from Python code as self.number.

For simple cases where a member variable is just read and written with no side effects, you can also declare getters and setters in your Rust struct field definition, for example:

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
#[pyclass]
struct MyClass {
  #[pyo3(get, set)]
  num: i32
}
#}

Then it is available from Python code as self.num.

To define a Python compatible method, an impl block for your struct has to be annotated with the #[pymethods] attribute. PyO3 generates Python compatible wrappers for all functions in this block with some variations, like descriptors, class method static methods, etc.

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
# #[pyclass]
# struct MyClass {
#    num: i32,
# }
#
#[pymethods]
impl MyClass {

     fn method1(&self) -> PyResult<i32> {
        Ok(10)
     }

     fn set_method(&mut self, value: i32) -> PyResult<()> {
        self.num = value;
        Ok(())
     }
}
#}

Calls to these methods are protected by the GIL, so both &self and &mut self can be used. The return type must be PyResult<T> or T for some T that implements IntoPy<PyObject>; the latter is allowed if the method cannot raise Python exceptions.

A Python parameter can be specified as part of method signature, in this case the py argument gets injected by the method wrapper, e.g.

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
# #[pyclass]
# struct MyClass {
#    num: i32,
#    debug: bool,
# }

#[pymethods]
impl MyClass {
     fn method2(&self, py: Python) -> PyResult<i32> {
        Ok(10)
     }
}
#}

From the Python perspective, the method2 in this example does not accept any arguments.

To create a class method for a custom class, the method needs to be annotated with the #[classmethod] attribute.

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
# use pyo3::types::PyType;
# #[pyclass]
# struct MyClass {
#    num: i32,
#    debug: bool,
# }

#[pymethods]
impl MyClass {
     #[classmethod]
     fn cls_method(cls: &PyType) -> PyResult<i32> {
        Ok(10)
     }
}
#}

Declares a class method callable from Python.

• The first parameter is the type object of the class on which the method is called. This may be the type object of a derived class.
• The first parameter implicitly has type &PyType.
• For details on parameter-list, see the documentation of Method arguments section.
• The return type must be PyResult<T> or T for some T that implements IntoPy<PyObject>.

To create a static method for a custom class, the method needs to be annotated with the #[staticmethod] attribute. The return type must be T or PyResult<T> for some T that implements IntoPy<PyObject>.

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
# #[pyclass]
# struct MyClass {
#    num: i32,
#    debug: bool,
# }

#[pymethods]
impl MyClass {
     #[staticmethod]
     fn static_method(param1: i32, param2: &str) -> PyResult<i32> {
        Ok(10)
     }
}
#}

To specify a custom __call__ method for a custom class, the method needs to be annotated with the #[call] attribute. Arguments of the method are specified as for instance methods.

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
use pyo3::types::PyTuple;
# #[pyclass]
# struct MyClass {
#    num: i32,
#    debug: bool,
# }

#[pymethods]
impl MyClass {
     #[call]
     #[args(args="*")]
     fn __call__(&self, args: &PyTuple) -> PyResult<i32> {
        println!("MyClass has been called");
        Ok(self.num)
     }
}
#}

By default, PyO3 uses function signatures to determine which arguments are required. Then it scans the incoming args and kwargs parameters. If it can not find all required parameters, it raises a TypeError exception. It is possible to override the default behavior with the #[args(...)] attribute. This attribute accepts a comma separated list of parameters in the form of attr_name="default value". Each parameter has to match the method parameter by name.

Each parameter can be one of the following types:

• "*": var arguments separator, each parameter defined after "*" is a keyword-only parameter. Corresponds to python's def meth(*, arg1.., arg2=..).
• args="*": "args" is var args, corresponds to Python's def meth(*args). Type of the args parameter has to be &PyTuple.
• kwargs="**": "kwargs" receives keyword arguments, corresponds to Python's def meth(**kwargs). The type of the kwargs parameter has to be Option<&PyDict>.
• arg="Value": arguments with default value. Corresponds to Python's def meth(arg=Value). If the arg argument is defined after var arguments, it is treated as a keyword-only argument. Note that Value has to be valid rust code, PyO3 just inserts it into the generated code unmodified.

Example:

# #![allow(unused_variables)]
#fn main() {
# use pyo3::prelude::*;
use pyo3::types::{PyDict, PyTuple};
#
# #[pyclass]
# struct MyClass {
#    num: i32,
#    debug: bool,
# }
#
#[pymethods]
impl MyClass {
    #[args(arg1=true, args="*", arg2=10, args3="\"Hello\"", kwargs="**")]
    fn method(&self, arg1: bool, args: &PyTuple, arg2: i32, arg3: &str, kwargs: Option<&PyDict>) -> PyResult<i32> {
        Ok(1)
    }
}
#}

Python's object model defines several protocols for different object behavior, like sequence, mapping or number protocols. PyO3 defines separate traits for each of them. To provide specific Python object behavior, you need to implement the specific trait for your struct. Important note, each protocol implementation block has to be annotated with the #[pyproto] attribute.

The PyObjectProtocol trait provides several basic customizations.

To customize object attribute access, define the following methods:

• fn __getattr__(&self, name: FromPyObject) -> PyResult<impl IntoPy<PyObject>>
• fn __setattr__(&mut self, name: FromPyObject, value: FromPyObject) -> PyResult<()>
• fn __delattr__(&mut self, name: FromPyObject) -> PyResult<()>

Each method corresponds to Python's self.attr, self.attr = value and del self.attr code.

• fn __repr__(&self) -> PyResult<impl ToPyObject<ObjectType=PyString>>

• fn __str__(&self) -> PyResult<impl ToPyObject<ObjectType=PyString>>

  Possible return types for __str__ and __repr__ are PyResult<String> or PyResult<PyString>.

• fn __bytes__(&self) -> PyResult<PyBytes>

  Provides the conversion to bytes.

• fn __format__(&self, format_spec: &str) -> PyResult<impl ToPyObject<ObjectType=PyString>>

  Special method that is used by the format() builtin and the str.format() method. Possible return types are PyResult<String> or PyResult<PyString>.

• fn __richcmp__(&self, other: impl FromPyObject, op: CompareOp) -> PyResult<impl ToPyObject>

  Overloads Python comparison operations (==, !=, <, <=, >, and >=). The op argument indicates the comparison operation being performed. The return type will normally be PyResult<bool>, but any Python object can be returned. If other is not of the type specified in the signature, the generated code will automatically return NotImplemented.

• fn __hash__(&self) -> PyResult<impl PrimInt>

  Objects that compare equal must have the same hash value. The return type must be PyResult<T> where T is one of Rust's primitive integer types.

• fn __bool__(&self) -> PyResult<bool>

  Determines the "truthyness" of the object.

If your type owns references to other Python objects, you will need to integrate with Python's garbage collector so that the GC is aware of those references. To do this, implement the PyGCProtocol trait for your struct. It includes two methods __traverse__ and __clear__. These correspond to the slots tp_traverse and tp_clear in the Python C API. __traverse__ must call visit.call() for each reference to another Python object. __clear__ must clear out any mutable references to other Python objects (thus breaking reference cycles). Immutable references do not have to be cleared, as every cycle must contain at least one mutable reference. Example:

# #![allow(unused_variables)]
#fn main() {
extern crate pyo3;

use pyo3::prelude::*;
use pyo3::PyTraverseError;
use pyo3::gc::{PyGCProtocol, PyVisit};

#[pyclass]
struct ClassWithGCSupport {
    obj: Option<PyObject>,
}

#[pyproto]
impl PyGCProtocol for ClassWithGCSupport {
    fn __traverse__(&self, visit: PyVisit) -> Result<(), PyTraverseError> {
        if let Some(ref obj) = self.obj {
            visit.call(obj)?
        }
        Ok(())
    }

    fn __clear__(&mut self) {
        if let Some(obj) = self.obj.take() {
            // Release reference, this decrements ref counter.
            let gil = GILGuard::acquire();
            let py = gil.python();
            py.release(obj);
        }
    }
}
#}

Special protocol trait implementations have to be annotated with the #[pyproto] attribute.

It is also possible to enable GC for custom classes using the gc parameter of the pyclass attribute. i.e. #[pyclass(gc)]. In that case instances of custom class participate in Python garbage collection, and it is possible to track them with gc module methods. When using the gc parameter, it is required to implement the PyGCProtocol trait, failure to do so will result in an error at compile time:

#[pyclass(gc)]
struct GCTracked {} // Fails because it does not implement PyGCProtocol

Iterators can be defined using the PyIterProtocol trait. It includes two methods __iter__ and __next__:

• fn __iter__(slf: PyRefMut<Self>) -> PyResult<impl IntoPy<PyObject>>
• fn __next__(slf: PyRefMut<Self>) -> PyResult<Option<impl IntoPy<PyObject>>>

Returning Ok(None) from __next__ indicates that that there are no further items.

Example:

# #![allow(unused_variables)]
#fn main() {
extern crate pyo3;

use pyo3::prelude::*;
use pyo3::PyIterProtocol;

#[pyclass]
struct MyIterator {
    iter: Box<Iterator<Item = PyObject> + Send>,
}

#[pyproto]
impl PyIterProtocol for MyIterator {
    fn __iter__(slf: PyRefMut<Self>) -> PyResult<Py<MyIterator>> {
        Ok(slf.into())
    }
    fn __next__(mut slf: PyRefMut<Self>) -> PyResult<Option<PyObject>> {
        Ok(slf.iter.next())
    }
}
#}

TODO: Which traits to implement (basically PyTypeCreate: PyObjectAlloc + PyTypeInfo + PyMethodsProtocol + Sized) and what they mean.

Users should be able to define a #[pyclass] with or without #[pymethods], while PyO3 needs a trait with a function that returns all methods. Since it's impossible to make the code generation in pyclass dependent on whether there is an impl block, we'd need to implement the trait on #[pyclass] and override the implementation in #[pymethods], which is to the best of my knowledge only possible with the specialization feature, which can't be used on stable.

To escape this we use inventory, which allows us to collect impls from arbitrary source code by exploiting some binary trick. See inventory: how it works and pyo3_derive_backend::py_class::impl_inventory for more details.

Python::eval is a method to execute a Python expression and returns the evaluated value as PyAny.

use pyo3::prelude::*;
use pyo3::types::IntoPyDict;

fn main() -> Result<(), ()> {
    let gil = Python::acquire_gil();
    let py = gil.python();
    let result = py.eval("[i * 10 for i in range(5)]", None, None).map_err(|e| {
        e.print_and_set_sys_last_vars(py);
    })?;
    let res: Vec<i64> = result.extract().unwrap();
    assert_eq!(res, vec![0, 10, 20, 30, 40]);
    Ok(())
}

Python::run is a method to execute one or more Python statements. This method returns nothing, but you can get objects via locals dict.

You can also use the py_run! macro, which is a shorthand for Python::run. Since py_run! can cause panic, we recommend you to use this macro only for testing your Python extensions quickly.

use pyo3::{PyObjectProtocol, prelude::*, py_run};
#  fn main() {
#[pyclass]
struct UserData {
    id: u32,
    name: String,
}
#[pymethods]
impl UserData {
    fn as_tuple(&self) -> (u32, String) {
        (self.id, self.name.clone())
    }
}
#[pyproto]
impl PyObjectProtocol for UserData {
    fn __repr__(&self) -> PyResult<String> {
        Ok(format!("User {}(id: {})", self.name, self.id))
    }
}
let gil = Python::acquire_gil();
let py = gil.python();
let userdata = UserData {
    id: 34,
    name: "Yu".to_string(),
};
let userdata = PyRef::new(py, userdata).unwrap();
let userdata_as_tuple = (34, "Yu");
py_run!(py, userdata userdata_as_tuple, r#"
assert repr(userdata) == "User Yu(id: 34)"
assert userdata.as_tuple() == userdata_as_tuple
"#);
# }

PyModule also can execute Python codes by calling a function.

use pyo3::{prelude::*, types::{IntoPyDict, PyModule}};
#  fn main() -> PyResult<()> {
let gil = Python::acquire_gil();
let py = gil.python();
let activators = PyModule::from_code(py, "
def relu(x):
    return max(0.0, x)

def leaky_relu(x, slope=0.01):
    return x if x >= 0 else x * slope
", "activators.py", "activators")?;
let relu_result: f64 = activators.call1("relu", (-1.0,))?.extract()?;
assert_eq!(relu_result, 0.0);
let kwargs = [("slope", 0.2)].into_py_dict(py);
let lrelu_result: f64 = activators
    .call("leaky_relu", (-1.0,), Some(kwargs))?
    .extract()?;
assert_eq!(lrelu_result, -0.2);
# Ok(()) }

CPython has the infamous GIL (Global Interpreter Lock), which prevents developers from getting true parallelism when running pure Python code. With PyO3, you can release the GIL when executing Rust code to achieve true parallelism.

The Python::allow_threads method temporarily releases the GIL, thus allowing other Python threads to run.

impl Python {
    pub fn allow_threads<T, F>(self, f: F) -> T where F: Send + FnOnce() -> T {}
}

Let's take a look at our word-count example, where we have a wc_parallel function that utilizes the rayon crate to count words in parallel.

fn wc_parallel(lines: &str, search: &str) -> i32 {
    lines.par_lines()
         .map(|line| wc_line(line, search))
         .sum()
}

Then in the Python bridge, we have a function search exposed to the Python runtime which calls wc_parallel inside a closure passed to Python::allow_threads to enable true parallelism:

#[pymodule]
fn word_count(py: Python, m: &PyModule) -> PyResult<()> {

    #[pyfn(m, "search")]
    fn search(py: Python, path: String, search: String) -> PyResult<i32> {
        let mut file = File::open(path)?;
        let mut contents = String::new();
        file.read_to_string(&mut contents)?;

        let count = py.allow_threads(move || wc_parallel(&contents, &search));
        Ok(count)
    }

    Ok(())
}

Let's benchmark the word-count example to verify that we did unlock true parallelism with PyO3. We are using pytest-benchmark to benchmark three word count functions:

1. Pure Python version
2. Rust sequential version
3. Rust parallel version

The benchmark script can be found here, then we can run pytest tests to benchmark them.

On MacBook Pro (Retina, 15-inch, Mid 2015) the benchmark gives:

PyO3's attributes (#[pyclass], #[pymodule], etc.) are procedural macros, which means that they rewrite the source of the annotated item. You can view the generated source with the following command, which also expands a few other things:

cargo rustc --profile=check -- -Z unstable-options --pretty=expanded > expanded.rs; rustfmt expanded.rs

(You might need to install rustfmt if you don't already have it.)

You can also debug classic !-macros by adding -Z trace-macros:

cargo rustc --profile=check -- -Z unstable-options --pretty=expanded -Z trace-macros > expanded.rs; rustfmt expanded.rs

See cargo expand for a more elaborate version of those commands.

Valgrind is a tool to detect memory management bugs such as memory leaks.

You first need to install a debug build of Python, otherwise Valgrind won't produce usable results. In Ubuntu there's e.g. a python3-dbg package.

Activate an environment with the debug interpreter and recompile. If you're on Linux, use ldd with the name of your binary and check that you're linking e.g. libpython3.6dm.so.1.0 instead of libpython3.6m.so.1.0.

Download the suppressions file for cpython.

Run Valgrind with valgrind --suppressions=valgrind-python.supp ./my-command --with-options

The best start to investigate a crash such as an segmentation fault is a backtrace.

• Link against a debug build of python as described in the previous chapter
• Run gdb <my-binary>
• Enter r to run
• After the crash occurred, enter bt or bt full to print the stacktrace

PyO3 exposes much of Python's C API through the ffi module.

The C API is naturally unsafe and requires you to manage reference counts, errors and specific invariants yourself. Please refer to the C API Reference Manual and The Rustonomicon before using any function from that API.

Currently, #341 causes cargo test to fail with weird linking errors when the extension-module feature is activated. For now you can work around this by making the extension-module feature optional and running the tests with cargo test --no-default-features:

[dependencies.pyo3]
version = "0.8.1"

[features]
extension-module = ["pyo3/extension-module"]
default = ["extension-module"]

PyO3 uses a build script to determine the Python version and set the correct linker arguments. By default it uses the python3 executable. You can override the Python interpreter by setting PYTHON_SYS_EXECUTABLE, e.g., PYTHON_SYS_EXECUTABLE=python3.6.

Different linker arguments must be set for libraries/extension modules and binaries, which includes both standalone binaries and tests. (More specifically, binaries must be told where to find libpython and libraries must not link to libpython for manylinux compliance).

Since PyO3's build script can't know whether you're building a binary or a library, you have to activate the extension-module feature to get the build options for a library, or it'll default to binary.

If you have e.g. a library crate and a profiling crate alongside, you need to use optional features. E.g. you put the following in the library crate:

[dependencies]
pyo3 = "0.6"

[lib]
name = "hyperjson"
crate-type = ["rlib", "cdylib"]

[features]
default = ["pyo3/extension-module"]

And this in the profiling crate:

[dependencies]
my_main_crate = { path = "..", default-features = false }
pyo3 = "0.6"

On Linux/macOS you might have to change LD_LIBRARY_PATH to include libpython, while on windows you might need to set LIB to include pythonxy.lib (where x and y are major and minor version), which is normally either in the libs or Lib folder of a Python installation.

There are two ways to distribute your module as a Python package: the old, setuptools-rust, and the new, maturin. setuptools-rust needs some configuration files (setup.py, MANIFEST.in, build-wheels.sh, etc.) and external tools (docker, twine). maturin doesn't need any configuration files. It can not yet build sdist though (pyo3/maturin#2).

Cross compiling PyO3 modules is relatively straightforward and requires a few pieces of software:

• A toolchain for your target.
• The appropriate options in your Cargo .config for the platform you're targeting and the toolchain you are using.
• A Python interpreter that's already been compiled for your target.
• The headers that match the above interpreter.

See https://github.com/japaric/rust-cross for a primer on cross compiling Rust in general.

After you've obtained the above, you can build a cross compiled PyO3 module by setting a few extra environment variables:

• PYO3_CROSS_INCLUDE_DIR: This variable must be set to the directory containing the headers for the target's Python interpreter.
• PYO3_CROSS_LIB_DIR: This variable must be set to the directory containing the target's libpython DSO.

An example might look like the following (assuming your target's sysroot is at /home/pyo3/cross/sysroot and that your target is armv7):

export PYO3_CROSS_INCLUDE_DIR="/home/pyo3/cross/sysroot/usr/include"
export PYO3_CROSS_LIB_DIR="/home/pyo3/cross/sysroot/usr/lib"

cargo build --target armv7-unknown-linux-gnueabihf

Using PyPy is supported via cpyext.

Support is only provided for building rust extension for code running under PyPy. This means PyPy cannot be called from rust via cpyext. Note that there some differences in the ffi module between pypy and cpython.

This is a limitation of cpyext and supported for embedding cpyext is not planned.

Compilation against PyPy is done by exporting the PYTHON_SYS_EXECUTABLE to a pypy binary or by compiling in a PyPy virtualenv.

For example, PYTHON_SYS_EXECUTABLE="/path/to/pypy3" /path/to/pypy3 setup.py install

These are features currently supported by PyO3, but not yet implemented in cpyext.

• Complex number functions (_Py_c_sum, _Py_c_sum ..)
• Conversion to rust's i128, u128 types.
• PySequence_Count (which is used to count number of element in array)
• PyDict_MergeFromSeq2 (used in PyDict::from_sequence)

PyO3 began as fork of rust-cpython when rust-cpython wasn't maintained. Over the time PyO3 has become fundamentally different from rust-cpython.

This chapter is based on the discussion in PyO3/pyo3#55.

While rust-cpython has a macro based dsl for declaring modules and classes, PyO3 uses proc macros and specialization. PyO3 also doesn't change your struct and functions so you can still use them as normal Rust functions. The disadvantage is that specialization currently only works on nightly.

rust-cpython

py_class!(class MyClass |py| {
    data number: i32;
    def __new__(_cls, arg: i32) -> PyResult<MyClass> {
        MyClass::create_instance(py, arg)
    }
    def half(&self) -> PyResult<i32> {
        Ok(self.number(py) / 2)
    }
});

pyo3

# #![allow(unused_variables)]
#fn main() {
use pyo3::prelude::*;
use pyo3::PyRawObject;

#[pyclass]
struct MyClass {
   num: u32,
}

#[pymethods]
impl MyClass {
    #[new]
    fn new(obj: &PyRawObject, num: u32) {
        obj.init({
            MyClass {
                num,
            }
        });
    }

    fn half(&self) -> PyResult<u32> {
        Ok(self.num / 2)
    }
}
#}

All objects are owned by the PyO3 library and all APIs available with references, while in rust-cpython, you own python objects.

Here is an example of the PyList API:

rust-cpython

impl PyList {

   fn new(py: Python) -> PyList {...}

   fn get_item(&self, py: Python, index: isize) -> PyObject {...}
}

pyo3

impl PyList {

   fn new(py: Python) -> &PyList {...}

   fn get_item(&self, index: isize) -> &PyObject {...}
}

Because PyO3 allows only references to Python objects, all references have the GIL lifetime. So the owned Python object is not required, and it is safe to have functions like fn py<'p>(&'p self) -> Python<'p> {}.

rust-cpython requires a Python parameter for constructing a PyErr, so error handling ergonomics is pretty bad. It is not possible to use ? with Rust errors.

PyO3 on other hand does not require Python for constructing a PyErr, it is only required if you want to raise an exception in Python with the PyErr::restore() method. Due to various std::convert::From<E> for PyErr implementations for Rust standard error types E, propagating ? is supported automatically.