Repository navigation
Encoder Reuse Bug? #212
Description
Hello!
I have an issue with reusing Encoder instances.
For some reason decoding fails in some cases. ("Extra 224 of 233 byte(s)", "Insufficient data")
The bug does not happen when just using the default encode function.
I am able to reproduce the problem with this script: https://gist.github.com/turnerEP/852147711da016ebb22296f75dbf7360
I am kinda confident that this is not expected behavior.
Cheers!
Activity
ProTip commentedon Sep 1, 2022
Landed on this today as well doing perf testing sending 170k messages at a time between a client and server.
It's definitely the encoder too. It always errors on the 150k(th) message but it's not that specific message or the one prior.. It seems to get into a bad state based on everything that came before(or at least a subset that I have not identified).
Here is a hex encoding of the bad data the encoder generated which failed on decode:
fix #212
Merge pull request #213 from msgpack/gfx/fix-212
gfx commentedon Sep 2, 2022
@turnerEP @ProTip
Thank you for your report. I can reproduce it with your gist. The behavior was designated because
Encoder#encode()returned the shared reference of the internalArrayBufferof the encoder. That's why it was faster than a function version ofencode().However, I've confirmed the behavior was confusing and the results of misuse were hard to debug. So I've changed the behavior at v2.8.0 to return a copy of the internal buffer of the encoder.
If you'd like to have a faster encoder and know what you are doing, use
Encoder#encodeSharedRef(), which returns the shared reference of the encoder. The tests added in #213 might help you to understand the behavior.FWIW I can't decode what @ProTip reported, so please file another issue if your problem is not fixed yet.