pyo3

Module pycell

Source
Expand description

PyO3’s interior mutability primitive.

Rust has strict aliasing rules - you can either have any number of immutable (shared) references or one mutable reference. Python’s ownership model is the complete opposite of that - any Python object can be referenced any number of times, and mutation is allowed from any reference.

PyO3 deals with these differences by employing the Interior Mutability pattern. This requires that PyO3 enforces the borrowing rules and it has two mechanisms for doing so:

  • Statically it can enforce thread-safe access with the Python<'py> token. All Rust code holding that token, or anything derived from it, can assume that they have safe access to the Python interpreter’s state. For this reason all the native Python objects can be mutated through shared references.
  • However, methods and functions in Rust usually do need &mut references. While PyO3 can use the Python<'py> token to guarantee thread-safe access to them, it cannot statically guarantee uniqueness of &mut references. As such those references have to be tracked dynamically at runtime, using PyCell and the other types defined in this module. This works similar to std’s RefCell type.

§When not to use PyCell

Usually you can use &mut references as method and function receivers and arguments, and you won’t need to use PyCell directly:

use pyo3::prelude::*;

#[pyclass]
struct Number {
    inner: u32,
}

#[pymethods]
impl Number {
    fn increment(&mut self) {
        self.inner += 1;
    }
}

The #[pymethods] proc macro will generate this wrapper function (and more), using PyCell under the hood:

ⓘ
// The function which is exported to Python looks roughly like the following
unsafe extern "C" fn __pymethod_increment__(
    _slf: *mut pyo3::ffi::PyObject,
    _args: *mut pyo3::ffi::PyObject,
) -> *mut pyo3::ffi::PyObject {
    use :: pyo3 as _pyo3;
    _pyo3::impl_::trampoline::noargs(_slf, _args, |py, _slf| {
        let _cell = py
            .from_borrowed_ptr::<_pyo3::PyAny>(_slf)
            .downcast::<_pyo3::PyCell<Number>>()?;
        let mut _ref = _cell.try_borrow_mut()?;
        let _slf: &mut Number = &mut *_ref;
        _pyo3::impl_::callback::convert(py, Number::increment(_slf))
    })
}

§When to use PyCell

§Using pyclasses from Rust

However, we do need PyCell if we want to call its methods from Rust:

Python::with_gil(|py| {
    let n = Py::new(py, Number { inner: 0 })?;

    // We borrow the guard and then dereference
    // it to get a mutable reference to Number
    let mut guard: PyRefMut<'_, Number> = n.bind(py).borrow_mut();
    let n_mutable: &mut Number = &mut *guard;

    n_mutable.increment();

    // To avoid panics we must dispose of the
    // `PyRefMut` before borrowing again.
    drop(guard);

    let n_immutable: &Number = &n.bind(py).borrow();
    assert_eq!(n_immutable.inner, 1);

    Ok(())
})

§Dealing with possibly overlapping mutable references

It is also necessary to use PyCell if you can receive mutable arguments that may overlap. Suppose the following function that swaps the values of two Numbers:

#[pyfunction]
fn swap_numbers(a: &mut Number, b: &mut Number) {
    std::mem::swap(&mut a.inner, &mut b.inner);
}

When users pass in the same Number as both arguments, one of the mutable borrows will fail and raise a RuntimeError:

>>> a = Number()
>>> swap_numbers(a, a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  RuntimeError: Already borrowed

It is better to write that function like this:

ⓘ
#[pyfunction]
fn swap_numbers(a: &PyCell<Number>, b: &PyCell<Number>) {
    // Check that the pointers are unequal
    if !a.is(b) {
        std::mem::swap(&mut a.borrow_mut().inner, &mut b.borrow_mut().inner);
    } else {
        // Do nothing - they are the same object, so don't need swapping.
    }
}

See the guide for more information.

Structs§