[asio-users] Why causes many CLOSE_WAIT in boost asio app?

[陈抒]

Hi,

I posted one question a few days ago, why my app based on boost::asio
didn't accept new
connection<http://stackoverflow.com/questions/20300926/why-my-app-based-on-boostasio-didnt-accept-new-connection>And
I thought it was solved.
But it is not, today I can re-produce the error sometimes. After
stopping/starting the 2000 client apps which connect to my TCP server. In
some cases, many connections are shown as CLOSE_WAIT.
I close the connection manually using below code:
void CloseSocket() {
try {
socket.shutdown(tcp::socket::shutdown_both);
socket.close();
BOOST_LOG_TRIVIAL(warning) << "close the connection";
} catch (std::exception& e) {
BOOST_LOG_TRIVIAL(warning) << "thread id: " << this_thread::get_id()
<< " " << e.what();
}
}

Also set address to be reused
acceptor_.set_option(boost::asio::ip::tcp::acceptor::reuse_address(true));

Some people said it's not recommended that closing the socket manually, let
asio do it for you. But some people said you should call shutdown before
close the socket. I use the latter way, but the problem still exists.

I have posted the question on stackoverflow, but it might be the right
place to post here. To get an official answer.

Thanks.


Dean Chen
Best regards
http://blog.csdn.net/csfreebird

[Marat Abrarov]

Hi, Dean Chen!

Post by 陈抒
Some people said it's not recommended that closing the socket manually,
let asio do it for you. But some people said you should call shutdown
before close the socket. I use the latter way, but the problem still exists.

For Windows only: http://msdn.microsoft.com/en-us/library/windows/desktop/ms738547(v=vs.85).aspx
Don’t forget to read (interesting) comments.

Regards,
Marat Abrarov.

[陈抒]

Hi,
Thanks.
I read the doc. On windows, there are two options SO_LINGER and
SO_DONTLINGER for closesocket API, also there is a bug.
But my TCP server is always running on windows. Any way for helping me to
analyze my problem?

Dean Chen
Best regards
http://blog.csdn.net/csfreebird

Post by Marat Abrarov
Hi, Dean Chen!

Post by 陈抒
Some people said it's not recommended that closing the socket manually,
let asio do it for you. But some people said you should call shutdown
before close the socket. I use the latter way, but the problem still

exists.
http://msdn.microsoft.com/en-us/library/windows/desktop/ms738547(v=vs.85).aspx
Don’t forget to read (interesting) comments.
Regards,
Marat Abrarov.
------------------------------------------------------------------------------
Sponsored by Intel(R) XDK
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio

[陈抒]

sorry, typo, my TCP server is always running on Linux( Ubuntu 12.04.3)

Dean Chen
Best regards
http://blog.csdn.net/csfreebird

Post by 陈抒
Hi,
Thanks.
I read the doc. On windows, there are two options SO_LINGER and
SO_DONTLINGER for closesocket API, also there is a bug.
But my TCP server is always running on windows. Any way for helping me
to analyze my problem?
Dean Chen
Best regards
http://blog.csdn.net/csfreebird

Post by Marat Abrarov
Hi, Dean Chen!

Post by 陈抒
Some people said it's not recommended that closing the socket manually,
let asio do it for you. But some people said you should call shutdown
before close the socket. I use the latter way, but the problem still

exists.
http://msdn.microsoft.com/en-us/library/windows/desktop/ms738547(v=vs.85).aspx
Don’t forget to read (interesting) comments.
Regards,
Marat Abrarov.
------------------------------------------------------------------------------
Sponsored by Intel(R) XDK
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio

[Marat Abrarov]

Post by 陈抒
I read the doc. On windows, there are two options SO_LINGER
and SO_DONTLINGER for closesocket API, also there is a bug.
sorry, typo, my TCP server is always running on Linux (Ubuntu 12.04.3)

This doc (http://msdn.microsoft.com/en-us/library/windows/desktop/ms738547(v=vs.85).aspx) is mostly about graceful TCP connection close (not about linger options) and the way it describes is crossplatform (half close). I use it in echo_server (https://github.com/mabrarov/asio_samples):

On the close initiator side:
1. Wait until current write operation completes (this doesn't means that data are sent).
2. Call shutdown(SD_SEND/SHUT_WR).
3. Wait until read operation returns EOF.
4. Close socket.

On the other side:
1. Close starts if EOF is received.
2. Wait until current write operation completes.
3. Call shutdown(SD_SEND/SHUT_WR).
4. EOF was already received, so we can close the socket.

Combine these approaches into the single state machine and use it both on server and client sides.

Read fine "Effective TCP/IP Programming", Jon C. Snader - Tip 16: Understand the TCP Orderly Release Operation.

Regards,
Marat Abrarov.

[陈抒]

Thank you, Marat. After working on it a few hours. I find I made some
mistakes
1. forget to increase the open file size using ulimit -n command in start
service script
This prevent my TCP server from accepting new connection request

2. My tcp server are using Nginx server as its TCP reverse proxy server.
The Nginx will connect to my TCP servers for testing at a specific time
interval. Because these are testing connection, my TCP server will shutdown
them soon once it finds they don't provide valid login information.
Unfortunately, I configure the time interval very small, just one second.
So many connections are shutdown by TCP server.

I also reviewed my code carefully to make sure all sockects are closed by
my CloseSocket function. Today, I have not see CLOSE_WAIT status anymore.
I will keep an eye on this, once it happens again, I will get back to here.

About your suggestion, for my server, I want to close the connection ASAP
because the security reason. Once getting wrong message, the socket must be
shutdown quickly, so graceful way might be not meet this requirement.
On client side, I developed newlisp app to simulate real 2,000 devices.
That's not a C++ application. For now I do not know how to do this in
newlisp, but I will do if I learn it.

My application works not like echo server. Server can send message to
device at any time, device can do this too. I will post a new thread to ask
for all of you here, just want to know how have done with asio in your real
product.


Dean Chen
Best regards
http://blog.csdn.net/csfreebird

Post by Marat Abrarov

Post by 陈抒
I read the doc. On windows, there are two options SO_LINGER
and SO_DONTLINGER for closesocket API, also there is a bug.
sorry, typo, my TCP server is always running on Linux (Ubuntu 12.04.3)

This doc (
http://msdn.microsoft.com/en-us/library/windows/desktop/ms738547(v=vs.85).aspx)
is mostly about graceful TCP connection close (not about linger options)
and the way it describes is crossplatform (half close). I use it in
1. Wait until current write operation completes (this doesn't means that data are sent).
2. Call shutdown(SD_SEND/SHUT_WR).
3. Wait until read operation returns EOF.
4. Close socket.
1. Close starts if EOF is received.
2. Wait until current write operation completes.
3. Call shutdown(SD_SEND/SHUT_WR).
4. EOF was already received, so we can close the socket.
Combine these approaches into the single state machine and use it both on
server and client sides.
Understand the TCP Orderly Release Operation.
Regards,
Marat Abrarov.
------------------------------------------------------------------------------
Sponsored by Intel(R) XDK
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio

[Bjorn Reese]

In some cases, many connections are shown as CLOSE_WAIT.

CLOSE_WAIT means that one side has closed the socket, but the other has
not. So first you should verify that your CloseSocket() is called for
every socket.

Glancing the code you posted on SO, the life-time of the ClientType
objects is unclear to me. You call ClientType::StartJob() in
AfterAccept(), but once you leave AfterAccept() the c object will
be destructed (unless you have another shared_ptr referencing it
elsewhere.)

PS: Forget about shutdown() and reuse_address() for the moment. They
are not solutions to your current problem.

[陈抒]

Hi,
My ClientType is Connection class here, inherit from
enable_shared_from_this
template<class T>
class Connection: public boost::enable_shared_from_this<T> {
...
};

And its sub-class Sign implements StartJob function like so:

void Sign::StartJob() {
BOOST_LOG_TRIVIAL(info) << "object id: " << id_ << " start job";
// send init request first
Send(AppHolder::Instance().get_init_request());
}

void Sign::Send(boost::shared_ptr<Message> msg) {
strand_.post(bind(&Sign::DoWrite, shared_from_this(), msg));
}

I will review my code carefully to make sure every socket is closed by my
CloseSocket() method.







Dean Chen
Best regards
http://blog.csdn.net/csfreebird

Post by Bjorn Reese

In some cases, many connections are shown as CLOSE_WAIT.

CLOSE_WAIT means that one side has closed the socket, but the other has
not. So first you should verify that your CloseSocket() is called for
every socket.
Glancing the code you posted on SO, the life-time of the ClientType
objects is unclear to me. You call ClientType::StartJob() in
AfterAccept(), but once you leave AfterAccept() the c object will
be destructed (unless you have another shared_ptr referencing it
elsewhere.)
PS: Forget about shutdown() and reuse_address() for the moment. They
are not solutions to your current problem.
------------------------------------------------------------------------------
Sponsored by Intel(R) XDK
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
asio-users mailing list
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio