[asio-users] How to make sure the read order.

Discussion:

陈抒

2013-12-05 16:20:55 UTC

Hi,
In my TCP server, server can send message to client under one of
following situations
1. one user fills in some text and clicks button on web site
2. server reads a message from a client and realizes it must send response
to client

Client always sends heartbeat message to TCP server every 50 seconds, and
it expects to get the heartbeat response message from TCP server.

So the interaction between Client and Server is not as simple as
request/response model.

Reading a message from client consists of two steps:
1. read header part, 5 bytes, the latter 4 bytes indicate the body's length
2. read body part

How to make sure the read operation work in a correct order? I hope server
app can read all messages one by one.
Do you have any best practice for this? Use a read queue for this?

Dean Chen
Best regards
http://blog.csdn.net/csfreebird

陈抒

2013-12-05 16:31:53 UTC

Permalink

Forgot to tell you what problem I ran into.
The client sends message to server, below is the output of tcpdump,
23:50:33.836728 IP (tos 0x0, ttl 64, id 10437, offset 0, flags [DF], proto
TCP (6), length 99)
192.168.1.56.34899 > 192.168.1.55.7778: Flags [P.], cksum 0x1b1d
(correct), seq 43:90, ack 88, win 457, options [nop,nop,TS val 286676524
ecr 90177612], length 47
0x0000: 5254 007a cae4 5254 0071 cae8 0800 4500 RT.z..RT.q....E.
0x0010: 0063 28c5 4000 4006 8e10 c0a8 0138 c0a8 .c(***@.@......8..
0x0020: 0137 8853 1e62 1da1 7dbd b94f f215 8018 .7.S.b..}..O....
0x0030: 01c9 1b1d 0000 0101 080a 1116 562c 0560 ............V,.`
0x0040: 004c 0130 3032 4653 6130 3133 3130 3331 .L.002FSa0131031
0x0050: 3030 3030 3433 3532 6130 3931 3463 3538 00004352a0914c58
0x0060: 6338 3430 6562 6632 3565 3464 3632 3942 c840ebf25e4d629B
0x0070: 03 .

The header received from server app is wrong. It should be 004c 0130
30(above red text), but now it got
53 61 30 31 33(above blue text)

Dean Chen
Best regards
http://blog.csdn.net/csfreebird

Post by éæ
Hi,
In my TCP server, server can send message to client under one of
following situations
1. one user fills in some text and clicks button on web site
2. server reads a message from a client and realizes it must send response
to client
Client always sends heartbeat message to TCP server every 50 seconds,
and it expects to get the heartbeat response message from TCP server.
So the interaction between Client and Server is not as simple as
request/response model.
1. read header part, 5 bytes, the latter 4 bytes indicate the body's length
2. read body part
How to make sure the read operation work in a correct order? I hope
server app can read all messages one by one.
Do you have any best practice for this? Use a read queue for this?
Dean Chen
Best regards
http://blog.csdn.net/csfreebird

Roger Austin (Australia)

2013-12-05 22:36:04 UTC

Permalink

You shouldnât need a read queue. After a client connects and the serverâs connection-handler code initiates the first read, all subsequent reads from that connection should be initiated by the read handler, so ensuring the data is read in the order it was sent.

However, you do need a write queue if there is the possibility that you will want to issue a write while there is another write pending or in progress. In particular, if your heartbeat message is issued asynchronously to other messages then you definitely need a write queue; otherwise the data from the two messages can become interleaved. Possibly this is what is causing your server to receive an incorrect header.

From: éæ [mailto:***@gmail.com]
Sent: Friday, 6 December 2013 03:32
To: asio-***@lists.sourceforge.net
Subject: Re: [asio-users] How to make sure the read order.

Forgot to tell you what problem I ran into.
The client sends message to server, below is the output of tcpdump,
23:50:33.836728 IP (tos 0x0, ttl 64, id 10437, offset 0, flags [DF], proto TCP (6), length 99)
192.168.1.56.34899 > 192.168.1.55.7778: Flags [P.], cksum 0x1b1d (correct), seq 43:90, ack 88, win 457, options [nop,nop,TS val 286676524 ecr 90177612], length 47
0x0000: 5254 007a cae4 5254 0071 cae8 0800 4500 RT.z..RT.q....E.
0x0010: 0063 28c5 4000 4006 8e10 c0a8 0138 c0a8 .c(***@.@......8<mailto:***@.@......8>..
0x0020: 0137 8853 1e62 1da1 7dbd b94f f215 8018 .7.S.b..}..O....
0x0030: 01c9 1b1d 0000 0101 080a 1116 562c 0560 ............V,.`
0x0040: 004c 0130 3032 4653 6130 3133 3130 3331 .L.002FSa0131031
0x0050: 3030 3030 3433 3532 6130 3931 3463 3538 00004352a0914c58
0x0060: 6338 3430 6562 6632 3565 3464 3632 3942 c840ebf25e4d629B
0x0070: 03 .

The header received from server app is wrong. It should be 004c 0130 30(above red text), but now it got
53 61 30 31 33(above blue text)

Dean Chen
Best regards
http://blog.csdn.net/csfreebird

On Fri, Dec 6, 2013 at 12:20 AM, éæ <***@gmail.com<mailto:***@gmail.com>> wrote:
Hi,
In my TCP server, server can send message to client under one of following situations
1. one user fills in some text and clicks button on web site
2. server reads a message from a client and realizes it must send response to client

Client always sends heartbeat message to TCP server every 50 seconds, and it expects to get the heartbeat response message from TCP server.

So the interaction between Client and Server is not as simple as request/response model.

Reading a message from client consists of two steps:
1. read header part, 5 bytes, the latter 4 bytes indicate the body's length
2. read body part

How to make sure the read operation work in a correct order? I hope server app can read all messages one by one.
Do you have any best practice for this? Use a read queue for this?

Dean Chen
Best regards
http://blog.csdn.net/csfreebird

Scott Mueller

2013-12-05 22:23:17 UTC

Permalink

Hello Dean,

My applications will typically have a MessageStartHandler function, which extracts important formatting, such as the length of the message from the stream. I use this by passing it to an async_read call that gets my header length. I then perform another async read with a function like MessageContinueHandler, which tries to get the rest of the information for the message with more async_read calls. When I have a full message detected, I do another async_read using MessageStartHandler to get the header of the next message, and then I pass the completed message into a message processing queue to do what needs to be done.

Best regards,

M. Scott Mueller

From: éæ [mailto:***@gmail.com]
Sent: Thursday, December 05, 2013 8:21 AM
To: asio-***@lists.sourceforge.net
Subject: [asio-users] How to make sure the read order.

Hi,
In my TCP server, server can send message to client under one of following situations
1. one user fills in some text and clicks button on web site
2. server reads a message from a client and realizes it must send response to client

Client always sends heartbeat message to TCP server every 50 seconds, and it expects to get the heartbeat response message from TCP server.

So the interaction between Client and Server is not as simple as request/response model.

Reading a message from client consists of two steps:
1. read header part, 5 bytes, the latter 4 bytes indicate the body's length
2. read body part

How to make sure the read operation work in a correct order? I hope server app can read all messages one by one.
Do you have any best practice for this? Use a read queue for this?

Dean Chen
Best regards
http://blog.csdn.net/csfreebird

陈抒

2013-12-06 17:22:20 UTC

Permalink

Hello Roger Austin
I agree with you about the read order, do not need read queue. I have
used this way before,
read head first
read body in read head handler function
read head again in read body handler function
keep doing the above steps

But when I get back to work on TCP project after several months, I forget
this. :)
This way is simple and correct, it's best practice!
I tried to following this advice, but still see some issues I mentioned
at first. Not sure what cause this? My newlisp client or my c++ TCP server,
or Nginx TCP proxy? I will trace this, maybe log the output of tcmpdump and
analyze the log file carefully.

About sending message, I keep using write queue because asio
doesn't guarantee the byte order if I sent two or more messages quickly. I
got this from one question on stackoverflow before.

Now I got two best practices on asio, one is how to read message,
another one is how to write message.
Thank you.

Dean Chen
Best regards
http://blog.csdn.net/csfreebird

Post by Scott Mueller
Hello Dean,
My applications will typically have a MessageStartHandler function, which
extracts important formatting, such as the length of the message from the
stream. I use this by passing it to an async_read call that gets my header
length. I then perform another async read with a function like
MessageContinueHandler, which tries to get the rest of the information for
the message with more async_read calls. When I have a full message
detected, I do another async_read using MessageStartHandler to get the
header of the next message, and then I pass the completed message into a
message processing queue to do what needs to be done.
Best regards,
M. Scott Mueller
*Sent:* Thursday, December 05, 2013 8:21 AM
*Subject:* [asio-users] How to make sure the read order.
Hi,
In my TCP server, server can send message to client under one of following situations
1. one user fills in some text and clicks button on web site
2. server reads a message from a client and realizes it must send response to client
Client always sends heartbeat message to TCP server every 50 seconds,
and it expects to get the heartbeat response message from TCP server.
So the interaction between Client and Server is not as simple as request/response model.
1. read header part, 5 bytes, the latter 4 bytes indicate the body's length
2. read body part
How to make sure the read operation work in a correct order? I hope
server app can read all messages one by one.
Do you have any best practice for this? Use a read queue for this?
Dean Chen
Best regards
http://blog.csdn.net/csfreebird
------------------------------------------------------------------------------
Sponsored by Intel(R) XDK
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio

陈抒

2013-12-06 17:34:13 UTC

Permalink

Hi, Scott:
then I pass the completed message into a message processing queue to do
what needs to be done.
Because the bussiness computing is not heavy in my TCP server currently, I
just process the messages in where I receive immediately.
But I am interested in your way, one day my app might evolve more complex
than now.
I guess you are using producer-consumer pattern. Could you introduce more
detail about this?
What message queue are you using?
On the other side, do you have a thread pool to process the message, do
some business job?
How to send message response to client?
That's another best practice I am looking for. What's the best/better
thread model when using asio?

Thanks.

Dean Chen
Best regards
http://blog.csdn.net/csfreebird

Post by éæ
Hello Roger Austin
I agree with you about the read order, do not need read queue. I have
used this way before,
read head first
read body in read head handler function
read head again in read body handler function
keep doing the above steps
But when I get back to work on TCP project after several months, I
forget this. :)
This way is simple and correct, it's best practice!
I tried to following this advice, but still see some issues I mentioned
at first. Not sure what cause this? My newlisp client or my c++ TCP server,
or Nginx TCP proxy? I will trace this, maybe log the output of tcmpdump and
analyze the log file carefully.
About sending message, I keep using write queue because asio
doesn't guarantee the byte order if I sent two or more messages quickly. I
got this from one question on stackoverflow before.
Now I got two best practices on asio, one is how to read message,
another one is how to write message.
Thank you.
Dean Chen
Best regards
http://blog.csdn.net/csfreebird

Post by Scott Mueller
Hello Dean,
My applications will typically have a MessageStartHandler function, which
extracts important formatting, such as the length of the message from the
stream. I use this by passing it to an async_read call that gets my header
length. I then perform another async read with a function like
MessageContinueHandler, which tries to get the rest of the information for
the message with more async_read calls. When I have a full message
detected, I do another async_read using MessageStartHandler to get the
header of the next message, and then I pass the completed message into a
message processing queue to do what needs to be done.
Best regards,
M. Scott Mueller
*Sent:* Thursday, December 05, 2013 8:21 AM
*Subject:* [asio-users] How to make sure the read order.
Hi,
In my TCP server, server can send message to client under one of following situations
1. one user fills in some text and clicks button on web site
2. server reads a message from a client and realizes it must send response to client
Client always sends heartbeat message to TCP server every 50 seconds,
and it expects to get the heartbeat response message from TCP server.
So the interaction between Client and Server is not as simple as
request/response model.
1. read header part, 5 bytes, the latter 4 bytes indicate the body's length
2. read body part
How to make sure the read operation work in a correct order? I hope
server app can read all messages one by one.
Do you have any best practice for this? Use a read queue for this?
Dean Chen
Best regards
http://blog.csdn.net/csfreebird
------------------------------------------------------------------------------
Sponsored by Intel(R) XDK
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio