On first glance, PyO3 provides a huge number of different types that can be used to wrap or refer to Python objects. This page delves into the details and gives an overview of their intended meaning, with examples when each type is best used.
Since Python has no concept of ownership, and works solely with boxed objects, any Python object can be referenced any number of times, and mutation is allowed from any reference.
The situation is helped a little by the Global Interpreter Lock (GIL), which ensures that only one thread can use the Python interpreter and its API at the same time, while non-Python operations (system calls and extension code) can unlock the GIL. (See the section on parallelism for how to do that in PyO3.)
In PyO3, holding the GIL is modeled by acquiring a token of the type
Python<'py>, which serves three purposes:
- It provides some global API for the Python interpreter, such as
- It can be passed to functions that require a proof of holding the GIL,
- Its lifetime can be used to create Rust references that implicitly guarantee
holding the GIL, such as
The latter two points are the reason why some APIs in PyO3 require the
py: Python argument, while others don't.
The PyO3 API for Python objects is written such that instead of requiring a
mutable Rust reference for mutating operations such as
PyList::append, a shared reference (which, in turn, can only
be created through
Python<'_> with a GIL lifetime) is sufficient.
However, Rust structs wrapped as Python objects (called
pyclass types) usually
&mut access. Due to the GIL, PyO3 can guarantee thread-safe acces
to them, but it cannot statically guarantee uniqueness of
&mut references once
an object's ownership has been passed to the Python interpreter, ensuring
references is done at runtime using
PyCell, a scheme very similar to
Represents: a GIL independent reference to a Python object of unspecified type.
Used: Whenever you want to carry around references to "some" Python object, without caring about a GIL lifetime. For example, storing Python object references in a Rust struct that outlives the Python-Rust FFI boundary, or returning objects from functions implemented in Rust back to Python.
Can be cloned using Python reference counts with
&ConcreteType(which must be a Python native type):
Represents: a GIL independent reference to a Python object of known type.
This can be a Python native type (like
PyTuple), or a
implemented in Rust.
PyObject, but with a known inner type.
pyclasstypes implemented in Rust, you get a
PyCell(see below). For Python native types, mutating operations through PyO3's API don't require
PyObject is semantically equivalent to
Py<PyAny> and might be
merged with it in the future.
Represents: a Python object of unspecified type, restricted to a GIL
PyAny can only ever occur as a reference, usually
Used: Whenever you want to refer to some Python object only as long as
holding the GIL. For example, intermediate values and arguments to
pymethods implemented in Rust where any type is allowed.
Represents: a native Python object of known type, restricted to a GIL
lifetime just like
Used: Whenever you want to operate with native Python types while holding
the GIL. Like
PyAny, this is the most convenient form to use for function
arguments and intermediate values.
Represents: a reference to a Rust object (instance of
PyClass) which is
wrapped in a Python object. The cell part is an analog to stdlib's
RefCell to allow access to
Used: for accessing pure-Rust API of the instance (members and functions
&mut SomeType) while maintaining the aliasing rules of
Represents: reference wrapper types employed by
PyCell to keep track of
borrows, analog to
RefMut used by
Used: while borrowing a
PyCell. They can also be used with
on types like
PyAny to get a reference quickly.
This trait marks structs defined in Rust that are also usable as Python classes,
usually defined using the
This trait marks structs that mirror native Python types, such as