← 输入框|github.com|刷新
Skip to content

gh-83055: asyncio.Queue: putting items out of order when it is full - #112496

Closed
YvesDup wants to merge 91 commits into
python:mainfrom
YvesDup:gh83055-asyncio.Queue
Closed

YvesDup wants to merge 91 commits into
python:mainfrom
YvesDup:gh83055-asyncio.Queue

Conversation

@YvesDup

@YvesDup YvesDup commented Nov 28, 2023 •

Copy link
Copy Markdown
Contributor

Currently, when the queue is full, all putting tasks are added to the queue._putters deque, via a new future.
As soon as an item is removed from this queue, one future of the deque is set to done, and removed from the deque queue._putters. This future moving from deque to queue, is in transit, not referenced anywhere.

First, I suggest removing the future from queue._putters only after it had been woken up, not before in order to keep in the deque all pending and moving tasks.

Second, when put() is called, if the queue is full or if queue._putters is not empty , this item/task will be added to queue._putters.
When put_nowait() is called, if the queue is full or if queue._putters is not empty, we raise a new QueueueueFullWithPendingPutTasks exception derived from QueueueueFull.

The behavior of the get part should be identical.

Modified methods of Queue class :

  • put()
  • put_nowait()
  • get()
  • get_nowait()
  • _wakeup_next()

New methods of Queue class:

  • _put_and_wakeup_next()
  • _get_and_wakeup_next()

New exceptions of queues.py:

  • QueueFullWithPendingPutTasks
  • QueueEmptyWithPendingGetTasks

YvesDup and others added 30 commits May 28, 2021 18:04
Add a Barrier object to asyncio synchronized primitives
Change after run make patchcheck
run make patchcheck
Add comment when testing self._state
Change if to while instruction  in _block method
Refactoring all test names of BarrierTests to be more readable
Add a method cancel_coros
Refactoring and simplifyiing some tests
Correction of RGX_REPR
after run python.bat Tools\scripts\patchcheck.py
Add  Barrier object as a clone of threading.Barrier
Describe when  action is called
first feedbacks from Emmanuel Arias
Remove  "import functools"
Suggestion of Emmanuel Arias: Add sample and description about Barrier Class
bpo-43352 - Add a sample and description for Barrier Object

Minor updates as remove crlf
bpo-43352 Suppress ` in a comment of the example
…ythonGH-24903)

Add a class BarrierTests to test Barrier object
Run i17 tests ncluding one about Barrier does not accept loop parameter
…ythonGH-24903)

Add the Barrier object in Synchronization section
Add a link to a Barrier sample  in Examples sub-section
Add the BrokenBarrierError Exception in Exception section
…ythonGH-24903)

Add a complete documentation inspired from threading.Barrier
Add a sample  to show how use a Barrier Object
…ythonGH-24903)

Add a Barrier object as a clone from thraeding Barrier
Update of __all__ variable to include this new class and the related BrokenBarrierError exception
…ythonGH-24903)

Add a class BarrierTests to test Barrier object
Run 17 tests including one about Barrier does not accept loop parameter
@YvesDup

YvesDup commented Nov 28, 2023 •

Copy link
Copy Markdown
Contributor Author

Sorry, Problems with old commits.

@YvesDup YvesDup changed the title gh-83055: asyncio.Queue: putting items out of order when it is full ~~gh-83055: asyncio.Queue: putting items out of order when it is full~~ Nov 28, 2023
@YvesDup YvesDup changed the title ~~gh-83055: asyncio.Queue: putting items out of order when it is full~~ gh-83055: asyncio.Queue: putting items out of order when it is full Nov 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant