Expand description
Rust bindings to the Python interpreter.
PyO3 can be used to write native Python modules or run Python code and modules from Rust.
See the guide for a detailed introduction.
§PyO3’s object types
PyO3 has several core types that you should familiarize yourself with:
§The Python<'py>
object, and the 'py
lifetime
Holding the global interpreter lock (GIL) is modeled with the Python<'py>
token. Many
Python APIs require that the GIL is held, and PyO3 uses this token as proof that these APIs
can be called safely. It can be explicitly acquired and is also implicitly acquired by PyO3
as it wraps Rust functions and structs into Python functions and objects.
The Python<'py>
token’s lifetime 'py
is common to many PyO3 APIs:
- Types that also have the
'py
lifetime, such as theBound<'py, T>
smart pointer, are bound to the Python GIL and rely on this to offer their functionality. These types often have a.py()
method to get the associatedPython<'py>
token. - Functions which depend on the
'py
lifetime, such asPyList::new
, require aPython<'py>
token as an input. Sometimes the token is passed implicitly by taking aBound<'py, T>
or other type which is bound to the'py
lifetime. - Traits which depend on the
'py
lifetime, such asFromPyObject<'py>
, usually have inputs or outputs which depend on the lifetime. Adding the lifetime to the trait allows these inputs and outputs to express their binding to the GIL in the Rust type system.
§Python object smart pointers
PyO3 has two core smart pointers to refer to Python objects, Py<T>
and its GIL-bound
form Bound<'py, T>
which carries the 'py
lifetime. (There is also
Borrowed<'a, 'py, T>
, but it is used much more rarely).
The type parameter T
in these smart pointers can be filled by:
PyAny
, e.g.Py<PyAny>
orBound<'py, PyAny>
, where the Python object type is not known.Py<PyAny>
is so common it has a type aliasPyObject
.- Concrete Python types like
PyList
orPyTuple
. - Rust types which are exposed to Python using the
#[pyclass]
macro.
See the guide for an explanation of the different Python object types.
§PyErr
The vast majority of operations in this library will return PyResult<...>
.
This is an alias for the type Result<..., PyErr>
.
A PyErr
represents a Python exception. A PyErr
returned to Python code will be raised as a
Python exception. Errors from PyO3
itself are also exposed as Python exceptions.
§Feature flags
PyO3 uses feature flags to enable you to opt-in to additional functionality. For a detailed description, see the Features chapter of the guide.
§Default feature flags
The following features are turned on by default:
macros
: Enables various macros, including all the attribute macros.
§Optional feature flags
The following features customize PyO3’s behavior:
abi3
: Restricts PyO3’s API to a subset of the full Python API which is guaranteed by PEP 384 to be forward-compatible with future Python versions.auto-initialize
: ChangesPython::with_gil
to automatically initialize the Python interpreter if needed.extension-module
: This will tell the linker to keep the Python symbols unresolved, so that your module can also be used with statically linked Python interpreters. Use this feature when building an extension module.multiple-pymethods
: Enables the use of multiple#[pymethods]
blocks per#[pyclass]
. This adds a dependency on the inventory crate, which is not supported on all platforms.
The following features enable interactions with other crates in the Rust ecosystem:
anyhow
: Enables a conversion from anyhow’sError
type toPyErr
.chrono
: Enables a conversion from chrono’s structures to the equivalent Python ones.chrono-tz
: Enables a conversion from chrono-tz’sTz
enum. Requires Python 3.9+.either
: Enables conversions between Python objects and either’sEither
type.eyre
: Enables a conversion from eyre’sReport
type toPyErr
.hashbrown
: Enables conversions between Python objects and hashbrown’sHashMap
andHashSet
types.indexmap
: Enables conversions between Python dictionary and indexmap’sIndexMap
.num-bigint
: Enables conversions between Python objects and num-bigint’sBigInt
andBigUint
types.num-complex
: Enables conversions between Python objects and num-complex’sComplex
type.num-rational
: Enables conversions between Python’s fractions.Fraction and num-rational’s typesrust_decimal
: Enables conversions between Python’s decimal.Decimal and rust_decimal’sDecimal
type.serde
: Allows implementing serde’sSerialize
andDeserialize
traits forPy
<T>
for allT
that implementSerialize
andDeserialize
.smallvec
: Enables conversions between Python list and smallvec’sSmallVec
.
§Unstable features
nightly
: Uses#![feature(auto_traits, negative_impls)]
to defineUngil
as an auto trait.
§rustc
environment flags
PyO3 uses rustc
’s --cfg
flags to enable or disable code used for different Python versions.
If you want to do this for your own crate, you can do so with the pyo3-build-config
crate.
Py_3_7
,Py_3_8
,Py_3_9
,Py_3_10
: Marks code that is only enabled when compiling for a given minimum Python version.Py_LIMITED_API
: Marks code enabled when theabi3
feature flag is enabled.PyPy
- Marks code enabled when compiling for PyPy.
§Minimum supported Rust and Python versions
PyO3 supports the following software versions:
- Python 3.7 and up (CPython and PyPy)
- Rust 1.63 and up
§Example: Building a native Python module
PyO3 can be used to generate a native Python module. The easiest way to try this out for the
first time is to use maturin
. maturin
is a tool for building and publishing Rust-based
Python packages with minimal configuration. The following steps set up some files for an example
Python module, install maturin
, and then show how to build and import the Python module.
First, create a new folder (let’s call it string_sum
) containing the following two files:
Cargo.toml
[package]
name = "string-sum"
version = "0.1.0"
edition = "2021"
[lib]
name = "string_sum"
# "cdylib" is necessary to produce a shared library for Python to import from.
#
# Downstream Rust code (including code in `bin/`, `examples/`, and `tests/`) will not be able
# to `use string_sum;` unless the "rlib" or "lib" crate type is also included, e.g.:
# crate-type = ["cdylib", "rlib"]
crate-type = ["cdylib"]
[dependencies.pyo3]
version = "0.23.0-dev"
features = ["extension-module"]
src/lib.rs
use pyo3::prelude::*;
/// Formats the sum of two numbers as string.
#[pyfunction]
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
Ok((a + b).to_string())
}
/// A Python module implemented in Rust.
#[pymodule]
fn string_sum(m: &Bound<'_, PyModule>) -> PyResult<()> {
m.add_function(wrap_pyfunction!(sum_as_string, m)?)?;
Ok(())
}
With those two files in place, now maturin
needs to be installed. This can be done using
Python’s package manager pip
. First, load up a new Python virtualenv
, and install maturin
into it:
$ cd string_sum
$ python -m venv .env
$ source .env/bin/activate
$ pip install maturin
Now build and execute the module:
$ maturin develop
# lots of progress output as maturin runs the compilation...
$ python
>>> import string_sum
>>> string_sum.sum_as_string(5, 20)
'25'
As well as with maturin
, it is possible to build using setuptools-rust or
manually. Both offer more flexibility than maturin
but require further
configuration.
§Example: Using Python from Rust
To embed Python into a Rust binary, you need to ensure that your Python installation contains a shared library. The following steps demonstrate how to ensure this (for Ubuntu), and then give some example code which runs an embedded Python interpreter.
To install the Python shared library on Ubuntu:
sudo apt install python3-dev
Start a new project with cargo new
and add pyo3
to the Cargo.toml
like this:
[dependencies.pyo3]
version = "0.23.0-dev"
# this is necessary to automatically initialize the Python interpreter
features = ["auto-initialize"]
Example program displaying the value of sys.version
and the current user name:
use pyo3::prelude::*;
use pyo3::types::IntoPyDict;
use pyo3::ffi::c_str;
fn main() -> PyResult<()> {
Python::with_gil(|py| {
let sys = py.import("sys")?;
let version: String = sys.getattr("version")?.extract()?;
let locals = [("os", py.import("os")?)].into_py_dict(py);
let code = c_str!("os.getenv('USER') or os.getenv('USERNAME') or 'Unknown'");
let user: String = py.eval(code, None, Some(&locals))?.extract()?;
println!("Hello {}, I'm Python {}", user, version);
Ok(())
})
}
The guide has a section with lots of examples about this topic.
§Other Examples
The PyO3 README contains quick-start examples for both using Rust from Python and Python from Rust.
The PyO3 repository’s examples subdirectory contains some basic packages to demonstrate usage of PyO3.
There are many projects using PyO3 - see a list of some at https://github.com/PyO3/pyo3#examples.
Re-exports§
pub use crate::conversion::AsPyPointer;
pub use crate::conversion::FromPyObject;
pub use crate::conversion::IntoPy;
pub use crate::conversion::ToPyObject;
pub use crate::marker::Python;
pub use crate::pycell::PyRef;
pub use crate::pycell::PyRefMut;
pub use crate::pyclass::PyClass;
pub use crate::pyclass_init::PyClassInitializer;
pub use crate::type_object::PyTypeCheck;
pub use crate::type_object::PyTypeInfo;
pub use crate::types::PyAny;
pub use crate::class::*;
Modules§
PyBuffer
implementation- Conversions to and from chrono’s
Duration
,NaiveDate
,NaiveTime
,DateTime<Tz>
,FixedOffset
, andUtc
. - Old module which contained some implementation details of the
#[pyproto]
module. - Defines conversions between Rust and Python types.
- Python coroutine implementation, used notably when wrapping
async fn
with#[pyfunction]
/#[pymethods]
. - Exception and warning types defined by Python.
- Raw FFI declarations for Python’s C API.
- Conversions to and from indexmap’s
IndexMap
. - Runtime inspection of objects exposed to Python.
- Fundamental properties of objects tied to the Python interpreter.
- Support for the Python
marshal
format. - Conversions to and from num-rational types.
- Helper to convert Rust panics to Python exceptions.
- PyO3’s prelude.
- Contains types for working with Python objects that own the underlying data.
- PyO3’s interior mutability primitive.
PyClass
and related traits.- Contains initialization utilities for
#[pyclass]
. - Conversions to and from rust_decimal’s [
Decimal
] type. - Conversions to and from smallvec.
- Synchronization mechanisms based on the Python GIL.
- Python type object information
- Various types defined by the Python interpreter such as
int
,str
andtuple
.
Macros§
- Add the module to the initialization table in order to make embedded Python code to use it. Module name is the argument.
- Defines a new exception type.
- Defines a Rust type for an exception defined in Python code.
- Variant of
import_exception
that does not emit code needed to use the imported exception type as a GIL Ref. - Interns
text
as a Python string and stores a reference to it in static storage. - A convenient macro to execute a Python code snippet, with some local variables set.
- Wraps a Rust function annotated with
#[pyfunction]
. - wrap_
pyfunction_ bound Deprecated Wraps a Rust function annotated with#[pyfunction]
. - Returns a function that takes a
Python
instance and returns a Python module.
Structs§
- A borrowed equivalent to
Bound
. - A GIL-attached equivalent to
Py<T>
. - Error that indicates a failure to convert a PyAny to a more specific Python type.
- Error that indicates a failure to convert a PyAny to a more specific Python type.
- A GIL-independent reference to an object allocated on the Python heap.
- Represents a Python exception.
- Represents the major, minor, and patch (if any) versions of this interpreter.
Traits§
- Owned or borrowed gil-bound Python smart pointer
- Helper conversion trait that allows to use custom arguments for lazy exception construction.
- Python exceptions that can be converted to
PyErr
.
Functions§
- Prepares the use of Python in a free-threaded context.
- Executes the provided closure with an embedded Python interpreter.
Type Aliases§
- A commonly-used alias for
Py<PyAny>
. - Represents the result of a Python call.
Attribute Macros§
- A proc macro used to expose Rust structs and fieldless enums as Python objects.
- A proc macro used to expose Rust functions to Python.
- A proc macro used to expose methods to Python.
- A proc macro used to implement Python modules.