Discussion:
[asio-users] UDP - maximum receive size
Gheorghe Marinca
2012-08-29 16:33:13 UTC
Permalink
Hi,

I send requests to a name server for name resolution. It's not clear for me
how can I tell when the receiving of data on the client socket has ended.
When I call "async_receive " on the socket I begin reading data from
server. Will asio call my receive completed handler (along with data on my
buffer) only when it has
detected that all the datagram has been received (that is for *one call*
to async_receive function) or is it possible to get that handler called
multiple times as it happens with tcp (do I have to call async receive
multiple times in this case?)

Also what is the effect on setting option receive buffer size on asio ? Any
thoughts are appreciated.

Regards
-Ghita
Green, Cliff
2012-08-29 17:28:55 UTC
Permalink
This is not directly an Asio question, but a general TCP / UDP question. There are many references (books, online) that will help with these questions.

Datagrams are always received in one read, and Asio methods follow that same model. If the buffer / requested size is too small, the data will be truncated. This is different than TCP, which is a stream protocol.

Check out various non-Asio discussions about receive buffer size - I don't think Asio adds anything to the behavior.

Cliff


From: Gheorghe Marinca [mailto:***@gmail.com]
Sent: Wednesday, August 29, 2012 9:33 AM
To: asio-***@lists.sourceforge.net
Subject: [asio-users] UDP - maximum receive size

Hi,

I send requests to a name server for name resolution. It's not clear for me how can I tell when the receiving of data on the client socket has ended.
When I call "async_receive " on the socket I begin reading data from server. Will asio call my receive completed handler (along with data on my buffer) only when it has
detected that all the datagram has been received (that is for *one call* to async_receive function) or is it possible to get that handler called multiple times as it happens with tcp (do I have to call async receive multiple times in this case?)

Also what is the effect on setting option receive buffer size on asio ? Any thoughts are appreciated.

Regards
-Ghita
Gheorghe Marinca
2012-08-29 18:43:05 UTC
Permalink
Thanks for replay Cliff. Looked over other frameworks/languages and it
seems that receive generally gets the content of all UDP datagram and
throws an exception if the client buffer is too small.
I find this quite strange considering that a UDP datagram can be arbitrary
large - even if it will be fragmented to about 1500 bytes (or so) on most
networks
Post by Green, Cliff
This is not directly an Asio question, but a general TCP / UDP question.
There are many references (books, online) that will help with these
questions.****
** **
Datagrams are always received in one read, and Asio methods follow that
same model. If the buffer / requested size is too small, the data will be
truncated. This is different than TCP, which is a stream protocol.****
** **
Check out various non-Asio discussions about receive buffer size – I don’t
think Asio adds anything to the behavior.****
** **
Cliff****
** **
** **
*Sent:* Wednesday, August 29, 2012 9:33 AM
*Subject:* [asio-users] UDP - maximum receive size****
** **
Hi,
I send requests to a name server for name resolution. It's not clear for
me how can I tell when the receiving of data on the client socket has ended.
When I call "async_receive " on the socket I begin reading data from
server. Will asio call my receive completed handler (along with data on my
buffer) only when it has
detected that all the datagram has been received (that is for *one call*
to async_receive function) or is it possible to get that handler called
multiple times as it happens with tcp (do I have to call async receive
multiple times in this case?)
Also what is the effect on setting option receive buffer size on asio ?
Any thoughts are appreciated.
Regards
-Ghita****
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio
Green, Cliff
2012-08-29 21:32:34 UTC
Permalink
Don't confuse underlying networking packet fragmentation with IP datagrams. Read up on layered protocols if that causes any confusion.

Datagrams can't be arbitrarily large - for IPv4 it's 65,535 bytes. I found this by searching: "IPv4 datagram maximum size" and Google found an answer for me in under a second. For IPv6 it is the same, unless it is a jumbogram. Again, Google is my friend. The Wikipedia entries even have a nice paragraph on packet fragmentation.

BTW, I assume you know that UDP is an unreliable protocol. Datagrams can be, and will be, dropped at any time for any reason by the sending or receiving OS, or any equipment / software in between.

Cliff

From: Gheorghe Marinca [mailto:***@gmail.com]
Sent: Wednesday, August 29, 2012 11:43 AM
To: asio-***@lists.sourceforge.net
Subject: Re: [asio-users] UDP - maximum receive size

Thanks for replay Cliff. Looked over other frameworks/languages and it seems that receive generally gets the content of all UDP datagram and throws an exception if the client buffer is too small.
I find this quite strange considering that a UDP datagram can be arbitrary large - even if it will be fragmented to about 1500 bytes (or so) on most networks

On Wed, Aug 29, 2012 at 8:28 PM, Green, Cliff <***@boeing.com<mailto:***@boeing.com>> wrote:
This is not directly an Asio question, but a general TCP / UDP question. There are many references (books, online) that will help with these questions.

Datagrams are always received in one read, and Asio methods follow that same model. If the buffer / requested size is too small, the data will be truncated. This is different than TCP, which is a stream protocol.

Check out various non-Asio discussions about receive buffer size - I don't think Asio adds anything to the behavior.

Cliff


From: Gheorghe Marinca [mailto:***@gmail.com<mailto:***@gmail.com>]
Sent: Wednesday, August 29, 2012 9:33 AM
To: asio-***@lists.sourceforge.net<mailto:asio-***@lists.sourceforge.net>
Subject: [asio-users] UDP - maximum receive size

Hi,

I send requests to a name server for name resolution. It's not clear for me how can I tell when the receiving of data on the client socket has ended.
When I call "async_receive " on the socket I begin reading data from server. Will asio call my receive completed handler (along with data on my buffer) only when it has
detected that all the datagram has been received (that is for *one call* to async_receive function) or is it possible to get that handler called multiple times as it happens with tcp (do I have to call async receive multiple times in this case?)

Also what is the effect on setting option receive buffer size on asio ? Any thoughts are appreciated.

Regards
-Ghita

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
asio-***@lists.sourceforge.net<mailto:asio-***@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio
Green, Cliff
2012-08-29 21:58:40 UTC
Permalink
Let me be a bit more precise - datagrams can be, and will be, dropped at any time by the sending or receiving OS or any equipment / software in between, for reasons out of the control of the sending or receiving application.

Cliff

From: Green, Cliff
Sent: Wednesday, August 29, 2012 2:33 PM
To: asio-***@lists.sourceforge.net
Subject: Re: [asio-users] UDP - maximum receive size

Don't confuse underlying networking packet fragmentation with IP datagrams. Read up on layered protocols if that causes any confusion.

Datagrams can't be arbitrarily large - for IPv4 it's 65,535 bytes. I found this by searching: "IPv4 datagram maximum size" and Google found an answer for me in under a second. For IPv6 it is the same, unless it is a jumbogram. Again, Google is my friend. The Wikipedia entries even have a nice paragraph on packet fragmentation.

BTW, I assume you know that UDP is an unreliable protocol. Datagrams can be, and will be, dropped at any time for any reason by the sending or receiving OS, or any equipment / software in between.

Cliff

From: Gheorghe Marinca [mailto:***@gmail.com]
Sent: Wednesday, August 29, 2012 11:43 AM
To: asio-***@lists.sourceforge.net<mailto:asio-***@lists.sourceforge.net>
Subject: Re: [asio-users] UDP - maximum receive size

Thanks for replay Cliff. Looked over other frameworks/languages and it seems that receive generally gets the content of all UDP datagram and throws an exception if the client buffer is too small.
I find this quite strange considering that a UDP datagram can be arbitrary large - even if it will be fragmented to about 1500 bytes (or so) on most networks
On Wed, Aug 29, 2012 at 8:28 PM, Green, Cliff <***@boeing.com<mailto:***@boeing.com>> wrote:
This is not directly an Asio question, but a general TCP / UDP question. There are many references (books, online) that will help with these questions.

Datagrams are always received in one read, and Asio methods follow that same model. If the buffer / requested size is too small, the data will be truncated. This is different than TCP, which is a stream protocol.

Check out various non-Asio discussions about receive buffer size - I don't think Asio adds anything to the behavior.

Cliff


From: Gheorghe Marinca [mailto:***@gmail.com<mailto:***@gmail.com>]
Sent: Wednesday, August 29, 2012 9:33 AM
To: asio-***@lists.sourceforge.net<mailto:asio-***@lists.sourceforge.net>
Subject: [asio-users] UDP - maximum receive size

Hi,

I send requests to a name server for name resolution. It's not clear for me how can I tell when the receiving of data on the client socket has ended.
When I call "async_receive " on the socket I begin reading data from server. Will asio call my receive completed handler (along with data on my buffer) only when it has
detected that all the datagram has been received (that is for *one call* to async_receive function) or is it possible to get that handler called multiple times as it happens with tcp (do I have to call async receive multiple times in this case?)

Also what is the effect on setting option receive buffer size on asio ? Any thoughts are appreciated.

Regards
-Ghita

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
asio-***@lists.sourceforge.net<mailto:asio-***@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio
Gheorghe Marinca
2012-08-30 04:36:45 UTC
Permalink
Thanks for clarification Cliff. When I said arbitrary large I made an
overstatement indeed but I had in mind the documented ipv4 udp packet
header size of 16 bits (from tcp/IP illustrated vol 1)

What I found quite hard to deal with efficiently is that when implementing
a dns resolver that protocol can send "arbitrary" large responses and the
only way to handle that in a single request response sequence is to
allocate very large receive buffers in this case.
Don’t confuse underlying networking packet fragmentation with IP
datagrams. Read up on layered protocols if that causes any confusion.
Datagrams can’t be arbitrarily large – for IPv4 it’s 65,535 bytes. I
found this by searching: “IPv4 datagram maximum size” and Google found an
answer for me in under a second. For IPv6 it is the same, unless it is a
jumbogram. Again, Google is my friend. The Wikipedia entries even have a
nice paragraph on packet fragmentation.
BTW, I assume you know that UDP is an unreliable protocol. Datagrams can
be, and will be, dropped at any time for any reason by the sending or
receiving OS, or any equipment / software in between.
Cliff
Sent: Wednesday, August 29, 2012 11:43 AM
Subject: Re: [asio-users] UDP - maximum receive size
Thanks for replay Cliff. Looked over other frameworks/languages and it
seems that receive generally gets the content of all UDP datagram and
throws an exception if the client buffer is too small.
I find this quite strange considering that a UDP datagram can be
arbitrary large - even if it will be fragmented to about 1500 bytes (or so)
on most networks
This is not directly an Asio question, but a general TCP / UDP question.
There are many references (books, online) that will help with these
questions.
Datagrams are always received in one read, and Asio methods follow that
same model. If the buffer / requested size is too small, the data will be
truncated. This is different than TCP, which is a stream protocol.
Check out various non-Asio discussions about receive buffer size – I
don’t think Asio adds anything to the behavior.
Cliff
Sent: Wednesday, August 29, 2012 9:33 AM
Subject: [asio-users] UDP - maximum receive size
Hi,
I send requests to a name server for name resolution. It's not clear for
me how can I tell when the receiving of data on the client socket has ended.
When I call "async_receive " on the socket I begin reading data from
server. Will asio call my receive completed handler (along with data on my
buffer) only when it has
detected that all the datagram has been received (that is for *one
call* to async_receive function) or is it possible to get that handler
called multiple times as it happens with tcp (do I have to call async
receive multiple times in this case?)
Also what is the effect on setting option receive buffer size on asio ?
Any thoughts are appreciated.
Regards
-Ghita
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio
Yuri Timenkov
2012-09-01 12:27:30 UTC
Permalink
Then you can expand your buffer and resend request. Also you can find how
similar non-asio open software deals with such issues (e.g. bind server).

On Thu, Aug 30, 2012 at 8:36 AM, Gheorghe Marinca <
Post by Gheorghe Marinca
Thanks for clarification Cliff. When I said arbitrary large I made an
overstatement indeed but I had in mind the documented ipv4 udp packet
header size of 16 bits (from tcp/IP illustrated vol 1)
What I found quite hard to deal with efficiently is that when implementing
a dns resolver that protocol can send "arbitrary" large responses and the
only way to handle that in a single request response sequence is to
allocate very large receive buffers in this case.
Don’t confuse underlying networking packet fragmentation with IP
datagrams. Read up on layered protocols if that causes any confusion.
Datagrams can’t be arbitrarily large – for IPv4 it’s 65,535 bytes. I
found this by searching: “IPv4 datagram maximum size” and Google found an
answer for me in under a second. For IPv6 it is the same, unless it is a
jumbogram. Again, Google is my friend. The Wikipedia entries even have a
nice paragraph on packet fragmentation.
BTW, I assume you know that UDP is an unreliable protocol. Datagrams can
be, and will be, dropped at any time for any reason by the sending or
receiving OS, or any equipment / software in between.
Cliff
Sent: Wednesday, August 29, 2012 11:43 AM
Subject: Re: [asio-users] UDP - maximum receive size
Thanks for replay Cliff. Looked over other frameworks/languages and it
seems that receive generally gets the content of all UDP datagram and
throws an exception if the client buffer is too small.
I find this quite strange considering that a UDP datagram can be
arbitrary large - even if it will be fragmented to about 1500 bytes (or so)
on most networks
This is not directly an Asio question, but a general TCP / UDP question.
There are many references (books, online) that will help with these
questions.
Datagrams are always received in one read, and Asio methods follow that
same model. If the buffer / requested size is too small, the data will be
truncated. This is different than TCP, which is a stream protocol.
Check out various non-Asio discussions about receive buffer size – I
don’t think Asio adds anything to the behavior.
Cliff
Sent: Wednesday, August 29, 2012 9:33 AM
Subject: [asio-users] UDP - maximum receive size
Hi,
I send requests to a name server for name resolution. It's not clear for
me how can I tell when the receiving of data on the client socket has ended.
When I call "async_receive " on the socket I begin reading data from
server. Will asio call my receive completed handler (along with data on my
buffer) only when it has
detected that all the datagram has been received (that is for *one
call* to async_receive function) or is it possible to get that handler
called multiple times as it happens with tcp (do I have to call async
receive multiple times in this case?)
Also what is the effect on setting option receive buffer size on asio ?
Any thoughts are appreciated.
Regards
-Ghita
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio
Loading...