[asio-users] message (datagram) oriented tcp protocol with asio

[James Flemer]

I am curious if anyone has implemented anything within the ASIO framework
to handle a tcp socket like it was datagram socket. Many protocols
implemented on TCP are really a sequence of discrete messages. The chat
example in the ASIO docs is a perfect example, the tcp stream is always a
header saying how long the payload is followed by the payload itself. In
the "chat_session" object, the action for receiving a message is hard coded
into the handle_read_body() method (i.e. calling room_.deliver).

The design I am envisioning would be used something like this instead (very
brief, but hopefully enough to demonstrate the idea):

chat::socket chat(io_service); // the chat io object, like
asio::ip::udp::socket
chat::room room;
chat::message msg; // or could be a "MutableBufferSequence"
chat.async_receive(msg, boost::bind(&chat::room::deliver, &room, _1));

So rather than chat working "on top" of ASIO, we extend ASIO to support the
"chat socket" as an extension of ASIO.

I think this could be "faked" by making chat::socket a "thick"
implementation that simply wraps multiple use of asio::ip::tcp similar to
how the current chat example is written. I believe this would require
handler storage or "chaining", where the handler used for the tcp socket
async_receive() was given the handler passed into
chat::scocket::async_receive(). But this doesn't really seem to be the
ASIO way.

I think, the "right" way would be to go ahead and implement a new service
as well that actually ended up using io_service::post to invoke the handler
given to chat::async_receive().

Has this been done? Does it sound like it would be a good idea to do? I
think it could be generalized for the class of protocols with fixed length
headers, and probably for the class of delimited header/body protocols as
well. Then chat would be something like:

struct chat {
typedef basic_fixed_header_message_socket<chat, ip::tcp> socket;
static const size_t header_size = 128;
// plus something to get payload size after receipt of header
}

Such a template (or set of templates) could really cut down on the
repetitive receive header, receive payload cyclic dance that chat so
clearly demonstrates.

-James

[Marat Abrarov]

Hi. James.

Post by James Flemer
So rather than chat working "on top" of ASIO, we extend ASIO
to support the "chat socket" as an extension of ASIO.
I think this could be "faked" by making chat::socket a "thick"
implementation that simply wraps multiple use of asio::ip::tcp
similar to how the current chat example is written. I believe
this would require handler storage or "chaining", where the
handler used for the tcp socket async_receive() was given the
handler passed into chat::scocket::async_receive(). But this
doesn't really seem to be the ASIO way.
Has this been done? Does it sound like it would be a good idea
to do? I think it could be generalized for the class of
protocols with fixed length headers, and probably for the class
of delimited header/body protocols as well. Then chat would be

Splitting of the TCP stream into the frames (actually we are speaking about reading from stream frame by frame, frame at once) can be implemented (frame protocol "header with payload size + payload") by the asio::async_read free function with special CompletionCondition functor:
http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/reference/async_read/overload2.html.

But this is rarely (IMHO) used because of:
1. Often it is important to register every (!) async_read_some completion for timeout reasons (to register timeout and abort any subsequent operations, may be this can be achieved by asio::async_read free function too).
2. The most widely used mode is to read (from socket, e.g. from kernel space into the user space) as much as the buffer of application supports and then parse the read data to get some frames/messages (the last one may be incomplete or it may be (still) no complete messages in the buffer at all). This "mode" has to minimize kernel/user space switches (there are some articles about it in the internet).

Feel free to look at my nmea_client project (there are no timeouts there but they can be added... good idea) at https://github.com/mabrarov/asio_samples. I'd switched from the idea to create custom asio::io_object (with custom Asio service - e.g. Asio extension) for every session class (chat::socket) because ma::handler_storage class can help to create such sessions without implementing Asio extension for each session (chat::socket) class. The ma::nmea::cyclic_read_session (https://github.com/mabrarov/asio_samples/blob/master/include/ma/nmea/cyclic_read_session.hpp) reads from stream and splits it into the frames itself. The public interface of ma::nmea::cyclic_read_session is very similar (in the part of reading from serial port) to the your chat::socket - it can fill an iterator-based range of frames (for zero-copying reasons frames are actually smart pointers to the raw frames):

// Handler()(const boost::system::error_code&, std::size_t)
template <typename Handler, typename Iterator>
void async_read_some(Iterator begin, Iterator end, const Handler& handler);

Regards,
Marat Abrarov.

[James Flemer]

Post by Marat Abrarov
Hi. James.

Post by James Flemer
So rather than chat working "on top" of ASIO, we extend ASIO
to support the "chat socket" as an extension of ASIO.
I think this could be "faked" by making chat::socket a "thick"
implementation that simply wraps multiple use of asio::ip::tcp
similar to how the current chat example is written. I believe
this would require handler storage or "chaining", where the
handler used for the tcp socket async_receive() was given the
handler passed into chat::scocket::async_receive(). But this
doesn't really seem to be the ASIO way.
Has this been done? Does it sound like it would be a good idea
to do? I think it could be generalized for the class of
protocols with fixed length headers, and probably for the class
of delimited header/body protocols as well. Then chat would be

Splitting of the TCP stream into the frames (actually we are speaking
about reading from stream frame by frame, frame at once) can be implemented
(frame protocol "header with payload size + payload") by the
http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/reference/async_read/overload2.html
.
1. Often it is important to register every (!) async_read_some completion
for timeout reasons (to register timeout and abort any subsequent
operations, may be this can be achieved by asio::async_read free function
too).
2. The most widely used mode is to read (from socket, e.g. from kernel
space into the user space) as much as the buffer of application supports
and then parse the read data to get some frames/messages (the last one may
be incomplete or it may be (still) no complete messages in the buffer at
all). This "mode" has to minimize kernel/user space switches (there are
some articles about it in the internet).

I had overlooked the asio::async_read free function with the
CompletionCondition functor. Thanks for pointing that out. It does seem
to handle a fixed-size header style protocol fairly well. I understand
your concern (#2) with performing small reads and thus many context
switches. I do not follow the concern (#1) about timeouts; I believe you
could still cancel() or close() the socket, though it might be tricky to
pass enough information (num bytes read) on for recovery.

As a proof of concept, I adapted the chat client example to use a templated
base class that performs the two-phase read using the async_read free
function. The chat client itself now makes a single async call with a
single handler that is invoked when the entire message is available. See:
https://github.com/jflemer/boost-chat-server
This is only a quick demo, I did not adapt the server part nor the writes.
There are some frustrating issues with boost bind passing the handler
through another handler; I used a mix of std::bind and boost::bind to get
around it, though there is probably a better way.

-James

[Marat Abrarov]

Hi, James.

There are some frustrating issues with boost bind passing the handler through another handler;
I used a mix of std::bind and boost::bind to get around it, though there is probably a better way.

Just use boost::protect (http://www.boost.org/doc/libs/1_55_0/libs/bind/bind.html#nested_binds) or tuples (https://github.com/mabrarov/asio_samples/blob/master/include/ma/nmea/cyclic_read_session.hpp, cyclic_read_session::start_extern_write_some & cyclic_read_session::handle_write, thanks to the author of Asio).

Regards,
Marat Abrarov.

[Bjorn Reese]

Post by James Flemer
I am curious if anyone has implemented anything within the ASIO
framework to handle a tcp socket like it was datagram socket. Many
protocols implemented on TCP are really a sequence of discrete messages.

Have a look at http:://axiomq.sourceforge.net/ which is a general
framework for building messaging protocols on top of ASIO. It introduces
a new basic socket class where the send and receive functions only
trigger a handler when the entire message has been sent/received. It
also introduces a new acceptor class, which handles any handshake or
negotiation that is required by the messaging protocol.

Currently, AxioMQ only supports a minimal implementation of the ZMTP/2.0
protocol (better known as ZeroMQ.) I did some experiments with HTTP to
ensure that AxioMQ can be used to implement WebSockets. Due to lack of
time, I have not made much progress though.

Post by James Flemer
I think this could be "faked" by making chat::socket a "thick"
implementation that simply wraps multiple use of asio::ip::tcp similar
to how the current chat example is written. I believe this would
require handler storage or "chaining", where the handler used for the
tcp socket async_receive() was given the handler passed into
chat::scocket::async_receive(). But this doesn't really seem to be the
ASIO way.

That is what I have done in AxioMQ, and it fits nicely with the ASIO
paradigm.

The only restriction is that the handler has to be a boost::function
(ASIO uses its own storage type underneath, but I do not know if I can
reuse it.)

Post by James Flemer
I think, the "right" way would be to go ahead and implement a new
service as well that actually ended up using io_service::post to invoke
the handler given to chat::async_receive().

Introducing a new io_service could make it more difficult to integrate
with other ASIO-enabled libraries.

For example, I have also written a Wake-on-LAN extension for ASIO
(which also uses the chaining approach) that I want to integrate with
my messaging sockets:

https://github.com/breese/awake