You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
An attempt to make asyncio.transport.writelines (selector) use Scatter I/O #84188
bpo-47010: Implement zero copy writes in SelectorSocketTransport in asyncio
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee=Noneclosed_at=<Date2022-03-17.13:11:02.294>created_at=<Date2020-03-18.21:00:01.366>labels= ['type-feature', '3.9', 'expert-asyncio']
title='An attempt to make asyncio.transport.writelines (selector) use Scatter I/O'updated_at=<Date2022-03-17.13:11:02.294>user='https://github.com/tzickel'
So I've attempted to write an version that uses sendmsg (scatter I/O) instead (will be attached in PR).
What I've learnt is:
It's hard to benchmark (If someone has an good example on checking if it's worth it, feel free to add such).
sendmsg has an OS limit on how many items can be done in one call. If the user does not call writer.drain() it might have too many items in the buffer, in that case I concat them (might be an expensive operation ? but that should not be th enormal case).
socket.socket.sendmsg can accept any bytes like iterable, but os.writev can only accept sequences, is that a bug ?
BTW, if wanted a much more simpler PR can be made, where writelines simply calls sendmsg on the input if no buffer exists, and if not only then concats and acts like the current code base.
Activity
tzickel commentedon Mar 18, 2020
I have a code that tries to be smart and prepare data to be chunked efficiently before sending, so I was happy to read about:
https://docs.python.org/3/library/asyncio-protocol.html#asyncio.WriteTransport.writelines
Only to see that it simply does:
So I've attempted to write an version that uses sendmsg (scatter I/O) instead (will be attached in PR).
What I've learnt is:
tzickel commentedon Mar 19, 2020
BTW, if wanted a much more simpler PR can be made, where writelines simply calls sendmsg on the input if no buffer exists, and if not only then concats and acts like the current code base.
asvetlov commentedon Mar 17, 2022
A duplicate of bpo-47010