[asio-users] Expected behavior of read with a streambuf and a completion routine

Discussion:

dawid_jurek

2016-05-14 17:10:54 UTC

Hello Philippe,

"The documentation says that read() results in zero or n call to read_some. In this case, the data are already in the buf...
Is it a bug ?"

No. It's expected behaviour. Yo are probably a little confused how boost::asio::read works (maybe because of boost::asio::read_until).
Boost::asio::read doesn't look at buffer content at all (but e.g. read_until does - it scans buffer to find delimeter occurence), it just
copy some number of bytes from socket buffer. Of course it requires network transfer so this call blocks.
Let's take a look on your example again:

boost::asio::streambuf buf;
// copy at least 2B from socket buffer to the end of buf
read(sock, buf, transfer_at_least(2));
buf.consume(2);
// again, copy at least next 2B from socket buffer to the end of buf
read(sock, buf, transfer_at_least(2));

Read can't work in a way you would like to because there is no "cooperation mechanism" between separated read-s.
Additional information like "recieved bytes so far" would be required to store somewhere in buf. So instead of this user must take care of counting
transffered bytes like below:

boost::asio::streambuf buf;
size_t recieved_bytes = read(sock, buf, transfer_at_least(2));
buf.consume(recieved_bytes);
recieved_bytes += read(sock, buf, transfer_at_least(2));

Anyway, backing to your original problem.
If you want to just read known data like "Welcome\n" as one message only one boost::asio::read (with transfer_at_least(sizeof("Welcome\n"))) call will be sufficient.

Regards,
Dawid

Hi,
I have encountered a strange behavior when reading on a TCP stream with a streambuf and a completion routine.
Imagine that I'm receiving in a single chunk: "Welcome\n".
streambuf buf;
read(sock, buf, at_least(2));
buf.consume(2);
read(sock, buf, at_least(2)); // Blocks
The documentation says that read() results in zero or n call to read_some. In this case, the data are already in the buf, but the completion routine is called with total_transferred = 0, resulting in a new call to read_some that blocks even if the data are already present.
Same behavior for async_read, but async/read_until returns directly with the data already in the buffer.
I was expecting that the data already in the streambuf was counted in the total_transferred parameter of the completion routine.
Is it a bug ?
Philippe Leuba
------------------------------------------------------------------------------
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio

------------------------------------------------------------------------------
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
_______________________________________________
asio-users mailing list
asio-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio

Sam Liddicott

2016-05-24 18:28:54 UTC

Permalink

I agree with all Dawid said, but I want to clarify for Philippe that there
is no need to call read a second time until he has consumed everything in
the buffer. In his case he would find that he could obtain the entire
welcome message without a need for the second read.

Calling read before emptying the buffer can be OK* if you are sure that
there will be more to read and append to the buffer.

In this case that is not true, hence the blocking.

Sam

* But not normally done with a blocking read

Post by dawid_jurek
Hello Philippe,
"The documentation says that read() results in zero or n call to
read_some. In this case, the data are already in the buf...
Is it a bug ?"
No. It's expected behaviour. Yo are probably a little confused how
boost::asio::read works (maybe because of boost::asio::read_until).
Boost::asio::read doesn't look at buffer content at all (but e.g.
read_until does - it scans buffer to find delimeter occurence), it just
copy some number of bytes from socket buffer. Of course it requires
network transfer so this call blocks.
boost::asio::streambuf buf;
// copy at least 2B from socket buffer to the end of buf
read(sock, buf, transfer_at_least(2));
buf.consume(2);
// again, copy at least next 2B from socket buffer to the end of buf
read(sock, buf, transfer_at_least(2));
Read can't work in a way you would like to because there is no
"cooperation mechanism" between separated read-s.
Additional information like "recieved bytes so far" would be required to
store somewhere in buf. So instead of this user must take care of counting
boost::asio::streambuf buf;
size_t recieved_bytes = read(sock, buf, transfer_at_least(2));
buf.consume(recieved_bytes);
recieved_bytes += read(sock, buf, transfer_at_least(2));
Anyway, backing to your original problem.
If you want to just read known data like "Welcome\n" as one message only
one boost::asio::read (with transfer_at_least(sizeof("Welcome\n"))) call
will be sufficient.
Regards,
Dawid

Hi,
I have encountered a strange behavior when reading on a TCP stream with

a streambuf and a completion routine.

Imagine that I'm receiving in a single chunk: "Welcome\n".
streambuf buf;
read(sock, buf, at_least(2));
buf.consume(2);
read(sock, buf, at_least(2)); // Blocks
The documentation says that read() results in zero or n call to

read_some. In this case, the data are already in the buf, but the
completion routine is called with total_transferred = 0, resulting in a new
call to read_some that blocks even if the data are already present.

Same behavior for async_read, but async/read_until returns directly with

the data already in the buffer.

I was expecting that the data already in the streambuf was counted in

the total_transferred parameter of the completion routine.

Is it a bug ?
Philippe Leuba

------------------------------------------------------------------------------

Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data

untouched!

https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio

------------------------------------------------------------------------------
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio

Philippe Leuba

2016-05-25 04:01:32 UTC

Permalink

Hi,

Thanks for your responses.

Yes, I understand now but I was a bit confused about the difference of behavior between read and read_until, so I can imagine others will be confused as well.

The documentation is not clear about this, even one of the given http server example application uses a second read without checking what is in the streambuf.

Imagine a protocol where the first n bytes give you the size of the complete message, quite common in stream oriented connection. You will typically need to do a read for at least n bytes, followed by a second read for the remaining bytes. Due to the read behavior, you will have to check if everything is already in the streambuf, and only if not, do an other read for the missing data. This is a bit tedious.

We can modify the first read to use an exactly completion condition, but this will be less efficient as this requires a specific OS read call.

I can always build the wanted behavior, on top of asio, but I was wondering in which case the pretty current low level behavior is really useful.

Regards

Philippe Leuba

Post by dawid_jurek
Hello Philippe,
"The documentation says that read() results in zero or n call to read_some. In this case, the data are already in the buf...
Is it a bug ?"
No. It's expected behaviour. Yo are probably a little confused how boost::asio::read works (maybe because of boost::asio::read_until).
Boost::asio::read doesn't look at buffer content at all (but e.g. read_until does - it scans buffer to find delimeter occurence), it just
copy some number of bytes from socket buffer. Of course it requires network transfer so this call blocks.
boost::asio::streambuf buf;
// copy at least 2B from socket buffer to the end of buf
read(sock, buf, transfer_at_least(2));
buf.consume(2);
// again, copy at least next 2B from socket buffer to the end of buf
read(sock, buf, transfer_at_least(2));
Read can't work in a way you would like to because there is no "cooperation mechanism" between separated read-s.
Additional information like "recieved bytes so far" would be required to store somewhere in buf. So instead of this user must take care of counting
boost::asio::streambuf buf;
size_t recieved_bytes = read(sock, buf, transfer_at_least(2));
buf.consume(recieved_bytes);
recieved_bytes += read(sock, buf, transfer_at_least(2));
Anyway, backing to your original problem.
If you want to just read known data like "Welcome\n" as one message only one boost::asio::read (with transfer_at_least(sizeof("Welcome\n"))) call will be sufficient.
Regards,
Dawid