Repository navigation
[RFC] _queue is missing #3608
Description
Summary
https://rustpython.github.io/pages/whats-left says _queue is missing. The module does not supply much apart from the native implementation. I would like to give it a try and replicate it here.
Detailed Explanation
Drawbacks, Rationale, and Alternatives
Not sure if it add any value, since the rustpython implementation seems good enough ?
Activity
youknowone commentedon Apr 14, 2022
Hi, welcome to the project. For now, it falls back to pure python implementation, which is also a part of official CPython library. If you add
_queue, it will enhance the performance. That's the value.There is already a working version and we lack of many other features, so it is not the first-grade prioritized, but at the same time, no reason to not to do. If you are motivated by that, go fot it! It will be an easy start point to dig in the project.
If you are looking for easy start point to learn how to start RustPython, I will also recommend this features: #3609
lsahn-gh commentedon May 15, 2026
@youknowone,
Can I take this if no one does ?
roryyorke commentedon May 31, 2026
SimpleQueue was added to CPython in python/cpython#3346 to address python/cpython#59181 .
The problem it solves is that queue.Queue is not guaranteed to be reentrant, and, at time of the issue, it was not (see https://codewithoutrules.com/2017/08/16/concurrency-python/ , linked in the CPython issue).
I've had a go at this, see https://github.com/roryyorke/RustPython/tree/rory/_queue , which passes the
CSimpleQueueTestsuite intest_queue.py; this suite includes a reentrancy test.I'd like to submit it for merge, but have some points for discussion first:
.clone()s. I added these during initial implementation because I thought they were needed for correct reference counting, but now I can't find the documentation that led me to think these calls were necessaryQueue.get(etc. etc.) don't have this property, so maybe this complexity is not needed nowparking_lot::Condvarwithrustpython_common::lock::PyMutex, but a recent-ish commit to main broke that, so I've used aparking_lot::Mutexinstead. I don't know the wider implications of this.Finally, while trying to understand the reentrancy idea I ran the example code in https://codewithoutrules.com/2017/08/16/concurrency-python/ with recent (3.13.5) CPython, and to my great surprise it did not deadlock. git bisection revealed that python/cpython@70af994 changed
threading.Condition.notifyso that queue.Queue, both in CPython and RustPython, no longer deadlock with this particular example.queue._PySimpleQueue, which uses athreading.Semaphore, does still deadlock with this example (again, in both CPython and RustPython) ---queue._PySimpleQueuehas a comment noting that it does not have the reentrant property.youknowone commentedon May 31, 2026
@roryyorke Thank you for the comment.
@ShaharNaveh could you follow-up this as author of #7994 ?
ShaharNaveh commentedon May 31, 2026
@roryyorke is very similar to an earlier implementation of #7994 https://github.com/ShaharNaveh/RustPython/blob/5501877c85230034dff29a0598bbdc2e1b4d5754/crates/stdlib/src/_queue.rs
That implementation has 1 major issue that it can be deadlocked by CPython test suite, specifically
test_multiprocessing_fork.test_manager. I am now testing things with ReentrantMutex to hopefully fix that potential deadlock.Also, there are some small stuff like making
weakref.ref(_queue.Empty)to pass, but that's a relatively easy fixyouknowone commentedon Oct 6, 2026
#7994 is merged, thank you all!