[asio-users] Are overlapping calls to async_write

Post by Florian Pflug
A) async_write_some() may be called again before a previous call has completed?

I think it is platform specific (depends on platform supported guarantees). At least at Windows it has to work: every
async_write_some/async_read_some forms exactly one IOCP operation (in case of usage of IOCP implementation).

Post by Florian Pflug
B) Assuming (A) for TCP sockets, might the data get interleaved arbitrarily?
(I'm interested in async_write_some() ONLY here - it seems obvious that
overlapping calls to composed operations like async_write() WOULD result
in pretty much arbitrary interleaving)

At Windows in practice WSASend (async_write_some in case of IOCP and TCP) ends in 2 cases:
1. Error happened.
2. All data were transferred.

I failed to find this in MSDN but it is confirmed by other Windows developers (I personally don't want to rely on what
MSDN keeps silent).

Post by Florian Pflug
C) Is my assumption correct that multiple async_wait() calls on a timer are allowed?

Yes, it is correct. Each timer has a queue of waiting handlers.

Post by Florian Pflug
D) How's the situation for async_read_some()?

In case of Windows in practice (not found in MSDN) WSARecv ends in 2 cases:
1. Error happened.
2. Some data were transferred.

May be someone can show me the MSDN confirmation for multiple simultaneous WSASend/WSARecv behavior?

Regards,
Marat Abrarov.

丘霖

2013-01-16 09:42:23 UTC

Post by Florian Pflug
A) async_write_some() may be called again before a previous call has completed?

You can do that, every time you call async_read_some(), the data will be
copied to send buffer or an error happens.
No matter what happens, the callback function will be called later.
Lower driver will try its best to copy data to buffer. So call
async_read_some() again before a previous has completed is not necessary.

Just don't do that simultaneously, the situation is same as A)

Post by Florian Pflug
C) Is my assumption correct that multiple async_wait() calls on a timer are allowed?

Correct!

Post by Florian Pflug
D) How's the situation for async_read_some()?

It makes no sense to do that on the same socket.

Marat Abrarov

2013-01-16 10:00:48 UTC

Post by Florian Pflug
A) async_write_some() may be called again before a previous call has completed?

You can do that, every time you call async_read_some(), the data will be copied to send buffer or an error happens.

I think it's not so simple. What can you say about situation when send buffer of socket is too small (for buffer
specified by asyn_write_some/WSASend) or set to 0-size?

Regards,
Marat Abrarov.

丘霖

2013-01-16 10:23:52 UTC

Post by ä¸é

Post by Florian Pflug
A) async_write_some() may be called again before a previous call has completed?

You can do that, every time you call async_read_some(), the data will be

copied to send buffer or an error happens.

Post by ä¸é
I think it's not so simple. What can you say about situation when send
buffer of socket is too small (for buffer
specified by asyn_write_some/WSASend) or set to 0-size?

The buffer will be filled with data. If the buffer is full, an error will
happen.

Marat Abrarov

2013-01-16 10:42:43 UTC

Post by Marat Abrarov
I think it's not so simple. What can you say about situation when send buffer of socket is too small (for buffer
specified by asyn_write_some/WSASend) or set to 0-size?

The buffer will be filled with data. If the buffer is full, an error will happen.

According to this logic I always get error for async_write_some with 0-size send buffer of socket. :)

丘霖

2013-01-16 11:25:38 UTC

Post by Marat Abrarov

Post by Marat Abrarov
I think it's not so simple. What can you say about situation when send

buffer of socket is too small (for buffer

Post by Marat Abrarov
specified by asyn_write_some/WSASend) or set to 0-size?
The buffer will be filled with data. If the buffer is full, an error

will happen.
According to this logic I always get error for async_write_some with
0-size send buffer of socket. :)
YES, by the way what's the result supposed to be in your opinion?

Marat Abrarov

2013-01-16 12:17:32 UTC

Post by Marat Abrarov

The buffer will be filled with data. If the buffer is full, an error will happen.

According to this logic I always get error for async_write_some with 0-size send buffer of socket. :)

According to this logic I always (even with the only outstanding async_write_some/WSASend operation!) get error for
async_write_some with 0-size send buffer of socket. But I use 0-size send buffer of socket (and 0-size receive buffer of
socket) and don't get errors.

Post by Marat Abrarov
YES, by the way what's the result supposed to be in your opinion?

For the 0-size send buffer of the socket: http://www.google.com/search?q=SO_SNDBUF+0+Windows.
The second result is the answer:
http://social.msdn.microsoft.com/Forums/ta/wsk/thread/4ff9ef97-5d3d-490f-8943-d58ca9751a8c.
So in this (0-size send buffer) case for the user of Winsock (and Asio) it looks like WSASend (async_write_some)
doesn't complete until user specified buffer is used (until operation is not completed by the kernel) i.e. until data
aren't sent physically.

As for the more than one simultaneous not completed WSASend operations (and not 0-size send buffer) - for the user of
Winsock it looks (maybe) similar - until Windows needs buffer it (Windows) wouldn't alert (user space) about operation
completion.

To the topic starter:
See http://msdn.microsoft.com/en-us/library/windows/desktop/ms742203%28v=vs.85%29.aspx.
It says:
"...
If you are using I/O completion ports, be aware that the order of calls made to WSASend is also the order in which the
buffers are populated. WSASend should not be called on the same socket simultaneously from different threads, because it
can result in an unpredictable buffer order.
..."
This can be treated as (at least) MSDN doesn't forbid more than one simultaneous not completed WSASend.

Post by Marat Abrarov
As far as I understand, the idea behind windows' overlapping I/O is that there need not
be any OS-maintained buffer at all. Instead, WSASend() simply queues the I/O request *without*
copying the data and delivers the completion notification once the data isn't needed anymore.
If there *is* a OS-maintained buffer, e.g. the socket's send buffer, the completion handler
is thus invoked once the data has been copied into that buffer. If there is *no* OS-maintained
buffer, e.g. if the socket's send buffer is 0-sized, the completion handler is instead invoked once the data has been

transmitted and acknowledged.

Post by Marat Abrarov
The question thus isn't so much how large the send buffer is, but whether there's a limit on the number of outstanding

I/O requests.

Yes, you are right. It is really so and it is really has limit. But this limit isn't described in MSDN.

Post by Marat Abrarov
And on unix-based systems, things seem to be quite different.
There, asio uses non-blocking sockets together with readability/writability notifications (i.e,

select/epoll/kqueue/whatever),

Post by Marat Abrarov
and the OS thus isn't aware of how many asio operations are outstanding. So on these systems,
the allowed number of overlapping async_write_some() calls should depend entirely on asio itself,
not on the OS. I don't yet understand the code well enough though to figure that number out...

Yes, you are right in this case too.

Regards,
Marat Abrarov.

Florian Pflug

2013-01-16 11:49:28 UTC

Post by Marat Abrarov

Post by Marat Abrarov
I think it's not so simple. What can you say about situation when send buffer of socket is too small (for buffer
specified by asyn_write_some/WSASend) or set to 0-size?

The buffer will be filled with data. If the buffer is full, an error will happen.

According to this logic I always get error for async_write_some with 0-size send buffer of socket. :)

As far as I understand, the idea behind windows' overlapping I/O is that there need not be any OS-maintained buffer at all. Instead, WSASend() simply queues the I/O request *without* copying the data and delivers the completion notification once the data isn't needed anymore. If there *is* a OS-maintained buffer, e.g. the socket's send buffer, the completion handler is thus invoked once the data has been copied into that buffer. If there is *no* OS-maintained buffer, e.g. if the socket's send buffer is 0-sized, the completion handler is instead invoked once the data has been transmitted and acknowledged.

The question thus isn't so much how large the send buffer is, but whether there's a limit on the number of outstanding I/O requests.

And on unix-based systems, things seem to be quite different. There, asio uses non-blocking sockets together with readability/writability notifications (i.e, select/epoll/kqueue/whatever), and the OS thus isn't aware of how many asio operations are outstanding. So on these systems, the allowed number of overlapping async_write_some() calls should depend entirely on asio itself, not on the OS. I don't yet understand the code well enough though to figure that number out...

Anyway, I think this whole area deserves more documentation.

best regards,
Florian Pflug

2013-01-16 12:39:27 UTC

Post by Florian Pflug
A) async_write_some() may be called again before a previous call has completed?

Certainly

Yes, it might, if sending data on the same socket. The docs for
async_write_some() read:
"The write operation may not transmit all of the data to the peer. Consider
using the async_write function if you need to ensure that all data is written
before the asynchronous operation completes."
If the second async_write_some() call is issued before the first one
completed,
and the first one failed to send all the data, then you get your interleaving.
This is the same interleaving that overlapping async_write() calls cause.
A sensible way to get around this problem is to have a queue of messages
and use the composite async_write() to send a single message only after the
previous one completed. Besides, if i am not mistaken async_write_some() is
allowed to return the would_block error, while async_write() reposts the
faild send internally instead.

with the the udp you have another problem on top of this.
even if all the sends are ordered correctly, the messages on the other side
pop up in the wrong order. Usually udp people,e.g. voip, add some kind of
sequence number to their messages so then the order can be corrected on
the receiving side.

Post by Florian Pflug
C) Is my assumption correct that multiple async_wait() calls on a timer are allowed?

Yes they are, but given there is only one deadline, what is the point in
having multiple handlers queued?

Post by Florian Pflug
D) How's the situation for async_read_some()?

the same, it is permitted to have multiple overlapping handlers, but it
could be of little benefit. For example, in multithreaded scenario, there is no
guarantee that the handler submitted first will be called the first when the
data arrives. Again, this is not just because of how boost.asio works.

some useful reading on the topic can be found in
http://www.serverframework.com/handling-multiple-pending-socket-read-and-
write-operations.html

Florian Pflug

2013-01-16 14:26:26 UTC

Post by Florian Pflug
A) async_write_some() may be called again before a previous call has completed?

Certainly

Just to be absolutely certain - that is true even if both calls affect the
*same* socket, and it works (per design, at least) on all platforms, right?

"The write operation may not transmit all of the data to the peer. Consider
using the async_write function if you need to ensure that all data is written
before the asynchronous operation completes."
If the second async_write_some() call is issued before the first one completed,
and the first one failed to send all the data, then you get your interleaving.
This is the same interleaving that overlapping async_write() calls cause.

Yeah, I'm aware of that. The question is, might the *actually sent* parts get
interleaved as well. I.e., say I do

socket.async_write_some(block1, on_first)
socket.async_write_some(block2, on_second)

and on_first() is told N1 bytes where sent while on_second() is told N2 bytes
where sent. Can I assume that the peer will receive block1[0..(N1-1)] followed
by block2[0..(N2-1)]? Or could it also be block2[0..(N2-1)] followed by
block1[0..(N1-1)], or even some arbitrary interleaving of block1 and block2?

On unix-based platforms, an arbitrary interleaving shouldn't be possible I think -
for that to happen, asio would need to do multiple write() calls per
call to async_write_some() which'd surprise me. But the two blocks might get
swapped if the socket is initially non-writable, and the second async_write_some
call gets the writability notification before the first one does.

On windows, I gather from the answers in this thread and from MSDN that the
ordering is guaranteed to be block1[0..(N1-1)] followed by block2[0..(N2-1)].

Post by vf
A sensible way to get around this problem is to have a queue of messages
and use the composite async_write() to send a single message only after the
previous one completed. Besides, if i am not mistaken async_write_some() is
allowed to return the would_block error, while async_write() reposts the
faild send internally instead.

Really? Where does it say that? I assumed that async_write_some() will call
the handler only if it either made some progress (i.e., sent at least a byte),
or a permanent error occurred (i.e., connection lost, socket invalid, …)

Post by Florian Pflug
C) Is my assumption correct that multiple async_wait() calls on a timer are allowed?

Yes they are, but given there is only one deadline, what is the point in
having multiple handlers queued?

It e.g. allows the following concise implementation of a periodic timer with
the ability to schedule handlers to run at the next tick.

periodic_timer::periodic_timer() {
reschedule();
}

periodic_timer::reschedule(error&) {
m_next = std::max(clock_type::now(), m_next + m_interval);
m_timer.expires_at(m_next);
m_timer.async_wait(bind(periodic_timer::reschedule, this));
}

periodic_timer::async_wait_tick(handler) {
m_timer.async_wait(handler);
}

Post by Florian Pflug
D) How's the situation for async_read_some()?

the same, it is permitted to have multiple overlapping handlers, but it
could be of little benefit. For example, in multithreaded scenario, there is no
guarantee that the handler submitted first will be called the first when the
data arrives. Again, this is not just because of how boost.asio works.
some useful reading on the topic can be found in
http://www.serverframework.com/handling-multiple-pending-socket-read-and-
write-operations.html

Interesting read, but seems to be windows-specific. One of the reasons I'm
using asio (and the reason I'm asking around here instead of just doing
experiments) is that I need to support Windows, Mac, Linux & iOS. Which is
why it's important for me to understand what's platform-specific behaviour,
and what's not.

BTW, once I've figured this out, I do plan to submit a documentation patch.

best regards,
Florian Pflug

2013-01-17 00:41:40 UTC

Post by Florian Pflug
A) async_write_some() may be called again before a previous call has completed?

Certainly

Just to be absolutely certain - that is true even if both calls affect the
*same* socket, and it works (per design, at least) on all platforms, right?

The answer is yes, see also below.

Yes, it might, if sending data on the same socket. The docs for
"The write operation may not transmit all of the data to the peer. Consider
using the async_write function if you need to ensure that all data is
written before the asynchronous operation completes."
If the second async_write_some() call is issued before the first one completed,
and the first one failed to send all the data, then you get your interleaving.
This is the same interleaving that overlapping async_write() calls cause.

No, see below.

Post by Florian Pflug
Or could it also be block2[0..(N2-1)] followed by
block1[0..(N1-1)],

Yes, see below.

or even some arbitrary interleaving of block1 and block2?

Life is random but not that random, see below.

Post by Florian Pflug
On unix-based platforms, an arbitrary interleaving shouldn't be possible I think -
for that to happen, asio would need to do multiple write() calls per
call to async_write_some() which'd surprise me. But the two blocks might
get swapped if the socket is initially non-writable, and the second a
sync_write_some
call gets the writability notification before the first one does.
On windows, I gather from the answers in this thread and from MSDN that the
ordering is guaranteed to be block1[0..(N1-1)]
followed by block2[0..(N2-1)].

imo this is a matter of principle, not the platform. When one calls send(),
they
submit a block to the driver. This involves a hard context switch, since
the driver runs in the kernel mode. The driver then sends whatever number
of bytes it can send over the wire and informs the caller that such and such
number of bytes have been sent.
It is the job of the driver to ensure that the portion of the the block
that it has reported as sent arrives to the other end in one piece.
This one of the good things about the tcp protocol, otherwise the internet
and other arguably helpful things would be impossible. Moreover, if two
blocks are submitted to the driver in a certain order, they arrive to the
other end in the same order (as mentioned, this is not the case for
udp though).

Now, the difference between WSASend() and send() is that WSASend() in async
mode does not induce a hard context switch.
Instead, it gently submits your block to the driver's
internal queue, and a kernel thread picks up the block when it feel like
it is not busy enough. This does not disrupt the OS thread scheduler,
avoids CPU stalls and generally better for the overall performance. The price
to pay for this is that the caller has no way knowing when their block is
submitted and how many bytes have been sent until after the async call
completes. So ordering calls to async_end_some() should not achieve much.

Intuitively, one could arrive to a conclusion, that this does not apply
on unix, because unix does not have WSASend().
Actually, it does, the posix standard specifies async IO just like in windows,
e.g
http://www.kernel.org/doc/man-pages/online/pages/man7/aio.7.html

However due the the sheer laziness (or lack of time, or incompetence?) of the
linux kernel team the 'official' advice is not to use aio_* APIs because
they are buggy.
Solaris, on the other hand, is unix too, but has very good set of
asynchronous APIs for both file and network IO.

Boost.Asio treats all posix platforms the same, by using epoll, and does a very
good job of emulating true asynchronous processing. So it would be reasonable
to assume that in terms of ordering of writes the behaviour of Boost.Asio
on posix platforms is no different to that on Windows. If they
fix the async IO in linux kernel, Boost.Asio may change its internals too, but
the library guarantees should stay.

I do not follow that. They don't impose any particular design pattern as people
have vastly different situations to deal with, threading, different protocols
etc. In the common instance of ..request-response-request-response... protocol
async_write() would do just fine.

Post by Florian Pflug
C) Is my assumption correct that multiple async_wait() calls on a timer are allowed?

Yes they are, but given there is only one deadline, what is the point in
having multiple handlers queued?

It e.g. allows the following concise implementation of a periodic timer with
the ability to schedule handlers to run at the next tick.
periodic_timer::periodic_timer() {
reschedule();
}
periodic_timer::reschedule(error&) {
m_next = std::max(clock_type::now(), m_next + m_interval);
m_timer.expires_at(m_next);
m_timer.async_wait(bind(periodic_timer::reschedule, this));
}
periodic_timer::async_wait_tick(handler) {
m_timer.async_wait(handler); <--

and when this is going to expire?

Post by Florian Pflug
}