fix(subinterpreter): don't touch the thread state before create() attaches one (#6127)
`subinterpreter::create()` documents that "the main interpreter and its GIL
are not required to be held prior to calling this function", but its first
statement is `error_scope err_scope;`, i.e. `PyErr_Fetch()`, before
`main_guard` attaches a thread state. With no current `PyThreadState`,
`PyErr_Fetch()` -> `_PyErr_GetRaisedException(NULL)` dereferences null and
the process dies (SIGSEGV; 0xC0000005 on Windows). `~error_scope` is the
mirror image: it calls `PyErr_Restore()` after `main_guard` has already
swapped the thread state back away.
Two ordinary situations reach `create()` with no thread state:
- an embedder that ends its initialization with `PyEval_SaveThread()`,
which is the documented way to hand the GIL back after
`Py_InitializeFromConfig()`;
- any worker thread that has never touched Python.
Existing tests never hit this because they all run under the
`py::scoped_interpreter guard{}` in catch.cpp, which keeps the GIL held on
the main thread for the whole run.
Move `error_scope` inside the `main_guard` scope. The case it exists for is
unaffected: a caller that already holds the main GIL takes
`subinterpreter_scoped_activate`'s `simple_gil_` fast path, which keeps the
same thread state, so its pending error is still saved across
`Py_NewInterpreterFromConfig()` and restored afterwards. A caller sitting on
some other interpreter never had its error indicator touched in the first
place, since everything inside the block runs on the main interpreter's
thread state and `PyThreadState_Swap()` does not move error indicators. It
also makes the `pybind11_fail()` path unwind in a safer order: `~error_scope`
now runs while `main_guard` is still alive.
Add "Create Subinterpreter without a thread state", covering both a thread
that dropped its thread state via `gil_scoped_release` and a thread that
never had one. It segfaults without the fix and passes with it.
Verified on Windows / MSVC 14.51 / CPython 3.13.14: test_with_catch goes
from 33 to 34 test cases, all passing.
Assisted-by: ClaudeCode:claude-opus-5diff --git a/include/pybind11/subinterpreter.h b/include/pybind11/subinterpreter.h
index def101f..f1e4e44 100644
--- a/include/pybind11/subinterpreter.h
+++ b/include/pybind11/subinterpreter.h
@@ -86,12 +86,15 @@
/// interpreter and its GIL are not required to be held prior to calling this function.
static subinterpreter create(PyInterpreterConfig const &cfg) {
- error_scope err_scope;
subinterpreter result;
{
// we must hold the main GIL in order to create a subinterpreter
subinterpreter_scoped_activate main_guard(main());
+ // error_scope reads and writes the *current* thread state, so it must be constructed
+ // only after main_guard has attached one (see PR #6127 for details).
+ error_scope err_scope;
+
auto *prev_tstate = PyThreadState_Get();
PyStatus status;
diff --git a/tests/test_with_catch/test_subinterpreter.cpp b/tests/test_with_catch/test_subinterpreter.cpp
index 3af100f..570519e 100644
--- a/tests/test_with_catch/test_subinterpreter.cpp
+++ b/tests/test_with_catch/test_subinterpreter.cpp
@@ -262,6 +262,54 @@
unsafe_reset_internals_for_single_interpreter();
}
+TEST_CASE("Create Subinterpreter without a thread state") {
+ // subinterpreter::create() documents that "the main interpreter and its GIL are not required
+ // to be held prior to calling this function". Embedders routinely end their initialization
+ // with PyEval_SaveThread(), which leaves the calling thread with no PyThreadState at all, and
+ // worker threads that have never touched Python have none either. So create() must not touch
+ // the current thread state before main_guard attaches one.
+
+ PyInterpreterState *main_interp = PyInterpreterState_Get();
+
+ {
+ py::gil_scoped_release nogil;
+ REQUIRE(py::detail::get_thread_state_unchecked() == nullptr);
+
+ // (a) on a thread that dropped its thread state
+ {
+ auto sub = py::subinterpreter::create();
+ REQUIRE(sub.id() >= 0);
+
+ {
+ py::subinterpreter_scoped_activate activate(sub);
+ REQUIRE(PyInterpreterState_Get() != main_interp);
+ }
+
+ REQUIRE(py::detail::get_thread_state_unchecked() == nullptr);
+ }
+
+ // (b) on a thread that never had one.
+ // REQUIRE throws on failure, so we can't use it within the thread: record what we see
+ // and check it on the main test thread after the join.
+ bool thread_started_without_tstate = false;
+ bool thread_result = false;
+ std::thread([&]() {
+ thread_started_without_tstate = (py::detail::get_thread_state_unchecked() == nullptr);
+
+ auto sub = py::subinterpreter::create();
+ py::subinterpreter_scoped_activate activate(sub);
+ thread_result = (PyInterpreterState_Get() != main_interp);
+ }).join();
+ REQUIRE(thread_started_without_tstate);
+ REQUIRE(thread_result);
+
+ REQUIRE(py::detail::get_thread_state_unchecked() == nullptr);
+ }
+
+ REQUIRE(PyInterpreterState_Get() == main_interp);
+ unsafe_reset_internals_for_single_interpreter();
+}
+
TEST_CASE("GIL Subinterpreter") {
PyInterpreterState *main_interp = PyInterpreterState_Get();