[asio-users] thread::interrupt and asio::ios_service::run

[kir]

Hello all,

Sorry if it is very basic, jsut could not find an answer. Does
io_service::run supports boost::thread::interrupt?

I wrote small example which i expected to work, but it does not
(prints "interrupting" and exits, second thread does not exit at all):

#include <boost/thread.hpp>
#include <boost/asio.hpp>
#include <iostream>

void hello_timer (boost::system::error_code const &ec) { std::cerr <<
"timer\n"; }

void ios_run (boost::asio::io_service *ios)
{
try
{
ios->run ();
std::cerr << "thread quit\n";
}
catch (std::exception const &e)
{
std::cerr << "exception: " + std::string (e.what ()) + "\n";
}
}

int main ()
{
boost::asio::io_service ios;
boost::asio::deadline_timer dt (ios);
dt.expires_from_now (boost::posix_time::minutes (15));
dt.async_wait (hello_timer);
boost::thread t (boost::bind (ios_run, &ios));
sleep (1);
std::cerr << "interrupting\n"; t.interrupt (); sleep (1);
return 0;
}

[Gruenke, Matt]

Not only would io_service need to support this, but so would every
handler you use, since the thread(s) calling io_service::run() are the
ones actually calling them.

I'd recommend directly calling io_service::stop(). Depending on your
aim, you might do even better to shut down all the state machines using
your io_service, causing it to exit simply by depriving it of work.


Matt


-----Original Message-----
From: kir
Sent: Tuesday, May 15, 2012 18:50
To: asio-***@lists.sourceforge.net
Subject: [asio-users] thread::interrupt and asio::ios_service::run

Hello all,

Sorry if it is very basic, jsut could not find an answer. Does
io_service::run supports boost::thread::interrupt?

I wrote small example which i expected to work, but it does not (prints
"interrupting" and exits, second thread does not exit at all):

#include <boost/thread.hpp>
#include <boost/asio.hpp>
#include <iostream>

void hello_timer (boost::system::error_code const &ec) { std::cerr <<
"timer\n"; }

void ios_run (boost::asio::io_service *ios) {
try
{
ios->run ();
std::cerr << "thread quit\n";
}
catch (std::exception const &e)
{
std::cerr << "exception: " + std::string (e.what ()) + "\n";
}
}

int main ()
{
boost::asio::io_service ios;
boost::asio::deadline_timer dt (ios);
dt.expires_from_now (boost::posix_time::minutes (15));
dt.async_wait (hello_timer);
boost::thread t (boost::bind (ios_run, &ios));
sleep (1);
std::cerr << "interrupting\n"; t.interrupt (); sleep (1);
return 0;
}

------------------------------------------------------------------------
------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and threat
landscape has changed and how IT managers can respond. Discussions will
include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
asio-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio

[kir]

On Wed, May 16, 2012 at 3:03 AM, Gruenke, Matt <***@tycoint.com> wrote:
> Not only would io_service need to support this, but so would every
> handler you use, since the thread(s) calling io_service::run() are the
> ones actually calling them.

So is it intended behavior? Is it so hard to support or is it a design
decision? (my handlers already support interruption, everything could
be just fine if io_service also did)

> I'd recommend directly calling io_service::stop().

I'll have to have global storage for them in my case or maybe one
global service. But anyway thanks for quick answer.

> Matt

As a side note i started from valgrind reporting invalid read around
epoll_reactor.ipp:364 (boost 1.47) which looks like this:

#if defined(BOOST_ASIO_HAS_TIMERFD)
bool check_timers = (timer_fd_ == -1);
#else // defined(BOOST_ASIO_HAS_TIMERFD)

But it has gone when i stripped example to minimum. I mean it may be a
design decision but still must not produce invalid reads.

[Gruenke, Matt]

I don't know if it's intended. I'm just saying the bar for correctly
implementing interruption support is a lot higher than stop. It's easy
to see, through testing & inspection, that stop() works correctly, but
that's much less true of interrupt(). If you don't need interrupt(),
I'd advise you to avoid it.

BTW, I assume the only reason someone would need interrupt() is to
seamlessly cancel threads, whether or not they're driving an
io_service() at the time. Otherwise, I'm puzzled. The primary benefit
of interrupt() is that it cancels blocking operation. However, since
the point of io_service is to perform nonblocking operations, it's not
clear to me what you hope to gain by it. If because your completion
handlers are blocking for significant amounts of time, you should be
aware that doing so may cause you other problems.


I can't answer for the valgrind problem you found. If you're able to
find the underlying cause (or at least verify that it's not a problem
with your test), please submit it to the boost.org bug tracker.


Matt


-----Original Message-----
From: kir
Sent: Tuesday, May 15, 2012 19:43
To: asio-***@lists.sourceforge.net
Subject: Re: [asio-users] thread::interrupt and asio::ios_service::run

On Wed, May 16, 2012 at 3:03 AM, Gruenke, Matt <***@tycoint.com>
wrote:
> Not only would io_service need to support this, but so would every
> handler you use, since the thread(s) calling io_service::run() are the

> ones actually calling them.

So is it intended behavior? Is it so hard to support or is it a design
decision? (my handlers already support interruption, everything could be
just fine if io_service also did)

> I'd recommend directly calling io_service::stop().

I'll have to have global storage for them in my case or maybe one global
service. But anyway thanks for quick answer.

> Matt

As a side note i started from valgrind reporting invalid read around
epoll_reactor.ipp:364 (boost 1.47) which looks like this:

#if defined(BOOST_ASIO_HAS_TIMERFD)
bool check_timers = (timer_fd_ == -1); #else //
defined(BOOST_ASIO_HAS_TIMERFD)

But it has gone when i stripped example to minimum. I mean it may be a
design decision but still must not produce invalid reads.

------------------------------------------------------------------------
------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and threat
landscape has changed and how IT managers can respond. Discussions will
include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
asio-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio

[kir]

On Wed, May 16, 2012 at 4:31 AM, Gruenke, Matt <***@tycoint.com> wrote:
> However, since the point of io_service is to perform nonblocking operations,
> it's not clear to me what you hope to gain by it.

i need to add asio-based server into other's person program wich
already has kind of cancel-all-threads-and-gracefully-exit code based
on thread::interrupt. I already used ::stop as you suggested although
it required non-local modifications.

> I can't answer for the valgrind problem... please submit it to the boost.org bug tracker.
>

sorry for that, looks like valgrind complained beyond main (in
destructor of global object), so i guess it is UB anyway. Will check
again later.

>
> Matt
>
>
> -----Original Message-----
> From: kir
> Sent: Tuesday, May 15, 2012 19:43
> To: asio-***@lists.sourceforge.net
> Subject: Re: [asio-users] thread::interrupt and asio::ios_service::run
>
> On Wed, May 16, 2012 at 3:03 AM, Gruenke, Matt <***@tycoint.com>
> wrote:
>> Not only would io_service need to support this, but so would every
>> handler you use, since the thread(s) calling io_service::run() are the
>
>> ones actually calling them.
>
> So is it intended behavior? Is it so hard to support or is it a design
> decision? (my handlers already support interruption, everything could be
> just fine if io_service also did)
>
>> I'd recommend directly calling io_service::stop().
>
> I'll have to have global storage for them in my case or maybe one global
> service. But anyway thanks for quick answer.
>
>> Matt
>
> As a side note i started from valgrind reporting invalid read around
> epoll_reactor.ipp:364 (boost 1.47) which looks like this:
>
> #if defined(BOOST_ASIO_HAS_TIMERFD)
>  bool check_timers = (timer_fd_ == -1); #else //
> defined(BOOST_ASIO_HAS_TIMERFD)
>
> But it has gone when i stripped example to minimum. I mean it may be a
> design decision but still must not produce invalid reads.
>
> ------------------------------------------------------------------------
> ------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and threat
> landscape has changed and how IT managers can respond. Discussions will
> include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> asio-users mailing list
> asio-***@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/asio-users
> _______________________________________________
> Using Asio? List your project at
> http://think-async.com/Asio/WhoIsUsingAsio
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> asio-users mailing list
> asio-***@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/asio-users
> _______________________________________________
> Using Asio? List your project at
> http://think-async.com/Asio/WhoIsUsingAsio

[Gruenke, Matt]

> i need to add asio-based server into other's person program wich
already has kind of
> cancel-all-threads-and-gracefully-exit code based on
thread::interrupt. I already
> used ::stop as you suggested although it required non-local
modifications.

What if you created a thread that existed simply to translate the
thread::interrupt() into io_service::stop()?

Something like this:

// global, for the sake of brevity
boost::asio::io_service IoSvc;

void IoSvc_threadmain()
{
IoSvc.run();
}

void IoSvc_glue_threadmain()
{
boost::thread iosvc_thread(IoSvc_threadmain);
try {
iosvc_thread.join();
}
catch (boost::thread_interrupted & exn)
{
IoSvc.stop();
iosvc_thread.join();
}
}


Then, the thread running IoSvc_glue_threadmain() is the only one visible
to the 3rd party code.


Matt


-----Original Message-----
From: kir
Sent: Tuesday, May 15, 2012 21:32
To: asio-***@lists.sourceforge.net
Subject: Re: [asio-users] thread::interrupt and asio::ios_service::run

On Wed, May 16, 2012 at 4:31 AM, Gruenke, Matt <***@tycoint.com>
wrote:
> However, since the point of io_service is to perform nonblocking
> operations, it's not clear to me what you hope to gain by it.

i need to add asio-based server into other's person program wich already
has kind of cancel-all-threads-and-gracefully-exit code based on
thread::interrupt. I already used ::stop as you suggested although it
required non-local modifications.

> I can't answer for the valgrind problem... please submit it to the
boost.org bug tracker.
>

sorry for that, looks like valgrind complained beyond main (in
destructor of global object), so i guess it is UB anyway. Will check
again later.

>
> Matt

[kir]

On Wed, May 16, 2012 at 7:43 PM, Gruenke, Matt <***@tycoint.com> wrote:

> What if you created a thread that existed simply to translate the
> thread::interrupt() into io_service::stop()?

That would work. But i will have extra thread per server object. It is
tolerable as this thread always sleeps and there is no memory limit. I
instead created global ioSvc and added stop to main just before exit
sequence is initiated. And your solution is local, so i will go with
it at some point. Thanks again. Still puzzled why asio does not
support interruption by itself.

[Gruenke, Matt]

> Still puzzled why asio does not support interruption by itself.

Just to be clear, *I* never said it didn't. Maybe it does with one narrow exception that you hit. I just tried to make my case against thread::interrupt(). I've never liked it, or pthread_cancel(), for that matter.


Matt


________________________________

From: kir
Sent: Thu 5/17/2012 1:18 AM
To: asio-***@lists.sourceforge.net
Subject: Re: [asio-users] thread::interrupt and asio::ios_service::run



On Wed, May 16, 2012 at 7:43 PM, Gruenke, Matt <***@tycoint.com> wrote:

> What if you created a thread that existed simply to translate the
> thread::interrupt() into io_service::stop()?

That would work. But i will have extra thread per server object. It is
tolerable as this thread always sleeps and there is no memory limit. I
instead created global ioSvc and added stop to main just before exit
sequence is initiated. And your solution is local, so i will go with
it at some point. Thanks again. Still puzzled why asio does not
support interruption by itself.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
asio-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio

[Dale Wilson]

Quoting "Gruenke, Matt" <***@Tycoint.com>:

> > Still puzzled why asio does not support interruption by itself.
>
> Just to be clear, *I* never said it didn't. Maybe it does with one narrow
> exception that you hit. I just tried to make my case against
> thread::interrupt(). I've never liked it, or pthread_cancel(), for that
> matter.
>
I think not liking it is a mild reaction. Hate or despise is a more appropriate
response. I would go so far as to say it is not possible to use it correctly.

The only safe thing to do after cancelling a thread is to kill the entire
process. Sure things may work by accident most of the time, but cancelling a
thread inherently leaves any resources that thread was using in an unstable
state.

Dale

[kir]

On Thu, May 17, 2012 at 5:46 PM, Dale Wilson <***@ociweb.com> wrote:
> The only safe thing to do after cancelling a thread is to kill the entire
> process.

which btw is my goal, i use interrupt for controlled cleanup and quit

> cancelling a
> thread inherently leaves any resources that thread was using in an unstable
> state.

That is wrong. There are defined cancelation points. If thread avoids
them there is no chance it will be cancelled. Also
boost::thread::interrupt results in exception, not cancellation in
sense of pthread_cancel.

[Gruenke, Matt]

Define cleanup. If you mean that you want to try and flush some buffers or something before calling exit(), then I can imagine cases where thread::interrupt() might be helpful. However, if you mean to shutdown in such a way that would satisfy memory leak checking tools, then you're likely to find that thread::interrupt() leads only down a road of dashed hopes, sorrows, and despair.

I think what Dale means is that because there are blocking calls which thread::interrupt() doesn't cancel, you can have a situation where thread A tears down resources used by thread B, which thread B later tries to use. This can result in a shutdown which nobody would consider "clean" (i.e. segfault, assertion failure, etc.).

Sorry, but in a non- garbage-collected language, there are no easy answers, nor substitutes for sorting out object ownership & shutdown sequences.


Matt


-----Original Message-----
From: kir
Sent: Thu 5/17/2012 11:51 AM
To: asio-***@lists.sourceforge.net
Subject: Re: [asio-users] thread::interrupt and asio::ios_service::run

On Thu, May 17, 2012 at 5:46 PM, Dale Wilson wrote:
> The only safe thing to do after cancelling a thread is to kill the entire
> process.

which btw is my goal, i use interrupt for controlled cleanup and quit

> cancelling a
> thread inherently leaves any resources that thread was using in an unstable
> state.

That is wrong. There are defined cancelation points. If thread avoids
them there is no chance it will be cancelled. Also
boost::thread::interrupt results in exception, not cancellation in
sense of pthread_cancel.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
asio-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio

[kir]

before too-much-text part. I asked about good way of cleanup because
in practice i saw several solutions and none of them seemed quite ok.
Most sane are these 3:

1. exit (0)
2. In destructor of each object there is a interrupt/join for each
object's "child" thread (worker thread, event loop thread, etc.)
3. there is a global thread_group which has all current threads, in
main group.interrupt_all is called, then group.join_all is called

way 1 meddles with valgrind (also process may crash with SIGSEGV on FreeBSD)

way 2 leads to unpredictable shutdown time and large amounts of
interrupt/join boilerplate

way 3 requires thread run via wrapper to ensure global thread group is
up to date, requires some cleanup sequences, looks clumsy

when asking my question i wanted to know if there exists proven way
for such seemingly common task as mt app shutdown.

further come comments for your mail, lengthy and perhaps not worth reading

On Sat, May 19, 2012 at 9:04 AM, Gruenke, Matt <***@tycoint.com> wrote:
> Define cleanup.

i need to
1. stop receiving connections
2. notify current network peers of termination and close sockets
3. store unfinished queries

i need it to be reasonably fast, few seconds

> However, if you mean to shutdown in
> such a way that would satisfy memory leak checking tools,

it is very desirable

> then you're likely
> to find that thread::interrupt() leads only down a road of dashed hopes,
> sorrows, and despair.

aren't you over-dramatizing? This is just a function, it works, what
is the problem? It is much simpler to use than pthread_cancel (which
is non-trivial even for C). After all threading library such as
boost::thread should provide mechanism to interrupt its own blocking
calls (mutex::lock, condition::wait, sleep), because there is no way
from outside to do it. And it provides such a call, thread::interrupt.
I don't really understand why you guys are so concerned about its
"inherently broken" and even "despicable" nature. I am not
boost::thread-orthodox and don't assume it to be canon. But i see no
conceptual flow in thread::interrupt. (and besides if there were so
deep conceptual problems in boost::thread::interrupt or
pthread_cancel, how they happened to stay in respective libraries for
so long?)

> I think what Dale means is that because there are blocking calls which
> thread::interrupt() doesn't cancel

These I plan to do all via asio one day. And in asio everything may be
canceled (via stop for now).

[Gruenke, Matt]

I'm not aware of a naive way to cleanly shutdown an arbitrarily-structured, multi-threaded C++ app. If you find one, please let me know.

I offered my recommendations and opinions in the spirit of helpfulness. I'll let you be the judge of whether our admonitions are overly-dramatic. I do not have a dog in this fight (I didn't write any part of Boost.Asio or Boost.Thread, though I have a fair amount of experience with each).

As I think this has gotten beyond the scope of Boost.Asio, I plan to refrain from further comment on this subject. You might have better luck in a broader forum, such as comp.lang.c++, stackoverflow.com, boost-users, etc. I wish you success in your travels.


Matt



-----Original Message-----
From: kir [mailto:***@gmail.com]
Sent: Sat 5/19/2012 7:41 PM
To: asio-***@lists.sourceforge.net
Subject: Re: [asio-users] thread::interrupt and asio::ios_service::run

before too-much-text part. I asked about good way of cleanup because
in practice i saw several solutions and none of them seemed quite ok.
Most sane are these 3:

1. exit (0)
2. In destructor of each object there is a interrupt/join for each
object's "child" thread (worker thread, event loop thread, etc.)
3. there is a global thread_group which has all current threads, in
main group.interrupt_all is called, then group.join_all is called

way 1 meddles with valgrind (also process may crash with SIGSEGV on FreeBSD)

way 2 leads to unpredictable shutdown time and large amounts of
interrupt/join boilerplate

way 3 requires thread run via wrapper to ensure global thread group is
up to date, requires some cleanup sequences, looks clumsy

when asking my question i wanted to know if there exists proven way
for such seemingly common task as mt app shutdown.

further come comments for your mail, lengthy and perhaps not worth reading

On Sat, May 19, 2012 at 9:04 AM, Gruenke, Matt <***@tycoint.com> wrote:
> Define cleanup.

i need to
1. stop receiving connections
2. notify current network peers of termination and close sockets
3. store unfinished queries

i need it to be reasonably fast, few seconds

> However, if you mean to shutdown in
> such a way that would satisfy memory leak checking tools,

it is very desirable

> then you're likely
> to find that thread::interrupt() leads only down a road of dashed hopes,
> sorrows, and despair.

aren't you over-dramatizing? This is just a function, it works, what
is the problem? It is much simpler to use than pthread_cancel (which
is non-trivial even for C). After all threading library such as
boost::thread should provide mechanism to interrupt its own blocking
calls (mutex::lock, condition::wait, sleep), because there is no way
from outside to do it. And it provides such a call, thread::interrupt.
I don't really understand why you guys are so concerned about its
"inherently broken" and even "despicable" nature. I am not
boost::thread-orthodox and don't assume it to be canon. But i see no
conceptual flow in thread::interrupt. (and besides if there were so
deep conceptual problems in boost::thread::interrupt or
pthread_cancel, how they happened to stay in respective libraries for
so long?)

> I think what Dale means is that because there are blocking calls which
> thread::interrupt() doesn't cancel

These I plan to do all via asio one day. And in asio everything may be

[Yuri Timenkov]

Hi Matt,

interrupt() doesn't lead to any resource leaks or missed cleanup. If you
look closer into documentation you'll see that interrupt() throws a
boost::thread_interrupted exception from well-defined cancellation points
which is caught in boost::thread's thread-main function. As with any other
exception stack will be unwound automatically and thread gracefully
finished.

If you write your program in exception-safe manner nothing leaks.

The problems may arise if you're unaware of these cancellation points.
Imagine you call join() from object destructor while one itself might be
executing in boost:thread. If someone interrupts this thread, you'll get
exception thrown from destructor which results in undefined behavior.

In other regards I'd like to see io_service::run as a cancellation point.

Best wishes,
Yuri

On Sat, May 19, 2012 at 9:04 AM, Gruenke, Matt <***@tycoint.com> wrote:

> **
>
> Define cleanup. If you mean that you want to try and flush some buffers
> or something before calling exit(), then I can imagine cases where
> thread::interrupt() might be helpful. However, if you mean to shutdown in
> such a way that would satisfy memory leak checking tools, then you're
> likely to find that thread::interrupt() leads only down a road of dashed
> hopes, sorrows, and despair.
>
> I think what Dale means is that because there are blocking calls which
> thread::interrupt() doesn't cancel, you can have a situation where thread A
> tears down resources used by thread B, which thread B later tries to use.
> This can result in a shutdown which nobody would consider "clean" (i.e.
> segfault, assertion failure, etc.).
>
> Sorry, but in a non- garbage-collected language, there are no easy
> answers, nor substitutes for sorting out object ownership & shutdown
> sequences.
>
>
> Matt
>
>
>
> -----Original Message-----
> From: kir
> Sent: Thu 5/17/2012 11:51 AM
> To: asio-***@lists.sourceforge.net
> Subject: Re: [asio-users] thread::interrupt and asio::ios_service::run
>
> On Thu, May 17, 2012 at 5:46 PM, Dale Wilson wrote:
> > The only safe thing to do after cancelling a thread is to kill the entire
> > process.
>
> which btw is my goal, i use interrupt for controlled cleanup and quit
>
> > cancelling a
> > thread inherently leaves any resources that thread was using in an
> unstable
> > state.
>
> That is wrong. There are defined cancelation points. If thread avoids
> them there is no chance it will be cancelled. Also
> boost::thread::interrupt results in exception, not cancellation in
> sense of pthread_cancel.
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> asio-users mailing list
> asio-***@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/asio-users
> _______________________________________________
> Using Asio? List your project at
> http://think-async.com/Asio/WhoIsUsingAsio
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> asio-users mailing list
> asio-***@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/asio-users
> _______________________________________________
> Using Asio? List your project at
> http://think-async.com/Asio/WhoIsUsingAsio
>
>

[Gruenke, Matt]

The concern I have with thread::interrupt() is the potential for it to introduce race conditions. There are simplistic cases, such as where you’re just using it on a single thread that only communicates with the cancelling thread, in which I agree that it’s not hard to use safely.



When discussing io_service::run(), it might help to think of it as a giant dispatch loop, as opposed to the sense in which the current cancellation points are like leaf nodes in the call graph. As I see it, making io_service::run() into a cancellation point is not a well-bounded problem.





Matt





From: Yuri Timenkov
Sent: Monday, May 21, 2012 04:30
To: asio-***@lists.sourceforge.net
Subject: Re: [asio-users] thread::interrupt and asio::ios_service::run



Hi Matt,

interrupt() doesn't lead to any resource leaks or missed cleanup. If you look closer into documentation you'll see that interrupt() throws a boost::thread_interrupted exception from well-defined cancellation points which is caught in boost::thread's thread-main function. As with any other exception stack will be unwound automatically and thread gracefully finished.

If you write your program in exception-safe manner nothing leaks.

The problems may arise if you're unaware of these cancellation points. Imagine you call join() from object destructor while one itself might be executing in boost:thread. If someone interrupts this thread, you'll get exception thrown from destructor which results in undefined behavior.

In other regards I'd like to see io_service::run as a cancellation point.

Best wishes,
Yuri

On Sat, May 19, 2012 at 9:04 AM, Gruenke, Matt <***@tycoint.com> wrote:

Define cleanup. If you mean that you want to try and flush some buffers or something before calling exit(), then I can imagine cases where thread::interrupt() might be helpful. However, if you mean to shutdown in such a way that would satisfy memory leak checking tools, then you're likely to find that thread::interrupt() leads only down a road of dashed hopes, sorrows, and despair.

I think what Dale means is that because there are blocking calls which thread::interrupt() doesn't cancel, you can have a situation where thread A tears down resources used by thread B, which thread B later tries to use. This can result in a shutdown which nobody would consider "clean" (i.e. segfault, assertion failure, etc.).

Sorry, but in a non- garbage-collected language, there are no easy answers, nor substitutes for sorting out object ownership & shutdown sequences.


Matt




-----Original Message-----
From: kir
Sent: Thu 5/17/2012 11:51 AM
To: asio-***@lists.sourceforge.net
Subject: Re: [asio-users] thread::interrupt and asio::ios_service::run

On Thu, May 17, 2012 at 5:46 PM, Dale Wilson wrote:
> The only safe thing to do after cancelling a thread is to kill the entire
> process.

which btw is my goal, i use interrupt for controlled cleanup and quit

> cancelling a
> thread inherently leaves any resources that thread was using in an unstable
> state.

That is wrong. There are defined cancelation points. If thread avoids
them there is no chance it will be cancelled. Also
boost::thread::interrupt results in exception, not cancellation in
sense of pthread_cancel.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
asio-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
asio-users mailing list
asio-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/asio-users
_______________________________________________
Using Asio? List your project at
http://think-async.com/Asio/WhoIsUsingAsio

[kir]

> As I see it, making
> io_service::run() into a cancellation point is not a well-bounded problem.
>

It is the same as implementing io_service::stop. Probably authors just
do not want to introduce boost::thread dependency, as there exists
standalone (non-boost) asio variant.

> Matt

[Gruenke, Matt]

Are you suggesting that thread::cancel() should have some special case
code to detect that the thread is calling io_service::run() and instead
call io_service::stop()? That seems feasible, although some users might
be dissatisfied to find that threads blocking on mutexes or condition
variables in their completion handlers aren't cancelled out of those
calls.

If not, I don't see how io_service::stop() and thread::cancel() are
equivalent, from an implementation perspective. The existing
cancellation points are all relatively small, self-contained functions.
In contrast, io_service::run() covers significant parts of the asio
codebase and call the user-specified completion handlers. In order to
correctly cancel a thread calling run() on an arbitrary io_service, all
of that code needs to correctly handle cancellation. And keep in mind
that there are blocking I/O operations that aren't cancellable. Those
are the kinds of challenges I have in mind, when I say it's not
well-bounded.

Also, I'm no expert on thread::cancel(), but it seems to me that a
cancelled thread can't synchronize with a worker thread it started (e.g.
via thread::join(), which is a cancellation point), which might be
necessary for clean & safe shutdown if the worker thread is using
resources owned by its parent. Even if there's a way to do this, it
still stands as an example of a case where code needs to be specifically
written to handle thread::cancel(), even if it's already written to be
exception-safe.


Matt


-----Original Message-----
From: kir
Sent: Wednesday, May 30, 2012 11:29
To: asio-***@lists.sourceforge.net
Subject: Re: [asio-users] thread::interrupt and asio::ios_service::run

> As I see it, making
> io_service::run() into a cancellation point is not a well-bounded
problem.
>

It is the same as implementing io_service::stop. Probably authors just
do not want to introduce boost::thread dependency, as there exists
standalone (non-boost) asio variant.

> Matt

[kir]

On Wed, May 30, 2012 at 8:21 PM, Gruenke, Matt <***@tycoint.com> wrote:
> Are you suggesting that thread::cancel() should have some special case
> code to detect that the thread is calling io_service::run() and instead
> call io_service::stop()?

yes, exactly

> That seems feasible, although some users might
> be dissatisfied to find that threads blocking on mutexes or condition
> variables in their completion handlers aren't cancelled out of those
> calls.

if boost::thread is consistently used, then all those calls are
cancellation points. Or do you mean non-boost mutexes?

> And keep in mind that there are blocking I/O operations that aren't cancellable.

what are those? At least for sockets I do not know of such operations.
Could you please explain?

Besides, be it thread::interrupt () or io_service::stop () those
operations are equally not cancellable. So we still have equivalent
implementation complexity for interrupt and stop. Not sure if i
understood you correctly though.

> Also, I'm no expert on thread::cancel(), but it seems to me that a
> cancelled thread can't synchronize with a worker thread it started (e.g.
> via thread::join(), which is a cancellation point), which might be
> necessary for clean & safe shutdown

boost::thread guys named method 'interrupt' not 'cancel' to highlight
conceptual difference. pthread_cancel cancels thread right away,
boost::thread::interrupt results in exception. During stack unwinding
we can synchronise the same way as we would do normally.

btw pthread_cancel provides cleanup callbacks wich also could be used
for syncing faik (not sure)

> if the worker thread is using resources owned by its parent.  Even if there's a way to do this, it
> still stands as an example of a case where code needs to be specifically
> written to handle thread::cancel(), even if it's already written to be
> exception-safe.

shared resources are almost always shared_ptr-ed anyway. I conclude
this from my practice so it is more a note than an argument.

[kir]

On Fri, Jun 1, 2012 at 9:37 PM, kir <***@gmail.com> wrote:
> On Wed, May 30, 2012 at 8:21 PM, Gruenke, Matt <***@tycoint.com> wrote:
>> Are you suggesting that thread::cancel() should have some special case
>> code to detect that the thread is calling io_service::run() and instead
>> call io_service::stop()?
>
> yes, exactly

oh sorry, not exactly.

If thread does io_service::run it must be interrupted same way as
io_service::stop would do, but without failing all pending operations.
Other threads could wake alongside with thread being interrupted.
Those must automatically restart (call ::run again). Then
boost::thread_interrupted exception must be thrown just as in other
interruption points.

real io_service::stop is called only when io_service destroyed.

[Roger Austin (Australia)]

Does io_service::stop fail (cancel) pending IO operations? I thought it just signalled the service to stop the run event loop that handles completed operations. If you want to cancel an IO operation, don't you have to do that yourself (e.g. shutdown a socket)? If you destroy the io_service object, that will cancel all pending operations (without calling their completion handlers).

If you are shutting down, why are you worried about cancelling pending IO operations? Isn't that what you want to do? The completion handlers for the cancelled operations will not be called after you call stop. The already-executing handlers will return when they are done (you can use interrupt to hurry them along if you like). All you have to do is join the threads and clean up (via use of shared_ptrs).

If a thread executing io_service::run throws an exception (including boost::thread_interrupted) that exception can propagate out of io_service::run without stopping the service. This is a deliberate design decision to allow the exception to be handled and the thread to rejoin the thread pool without interfering with the operation of other threads. If you want to stop the service when the thread is interrupted, catch thread_interrupted, call io_service::stop, join any child threads, and rethrow.

Generally speaking, I have found the clean way to shut down when using asio is to call io_service::stop, join with any threads that are executing io_service::run, release any shared_ptrs to io objects such as sockets, and destroy the service (which cancels any pending IO operations and cleans up any remaining handlers, including io objects bound to those handlers via shared_ptrs). This assumes you have one io_service or multiple independent io_services. If you have multiple io_services and io objects that refer to one io_service are bound to handlers for operations that run on a different io_service, then you run into order-of-destruction problems. Ideally you would be able to shutdown all the services before destroying them, but unfortunately the shutdown_service process is not publically accessible. The one time I tripped over this problem I was able to resolve it by destroying the services in a particular order.

Conceptually I don't see a problem with making io_service::run into a cancellation point, but it would require the boost and non-boost implementations to be different and might require different implementations on different operating systems (in so far as run and related methods have different implementations for different OSes).

Thread::interrupt sounds like a nice idea in principle, but unless all the threads in your app are boost threads, and all the blocking methods they call are synchronisation points, it can never be relied on. Any call into a third-party library would potentially break your shutdown process (as you have discovered with asio). If there is ever to be a reliable way of cleanly interrupting threads, it is going to require OS-level support to raise an OS-level exception on a thread and language-level support to translate that OS-level exception into a C++ exception.

________________________________________
From: kir [***@gmail.com]
Sent: Saturday, 2 June 2012 3:56 AM
To: asio-***@lists.sourceforge.net
Subject: Re: [asio-users] thread::interrupt and asio::ios_service::run

On Fri, Jun 1, 2012 at 9:37 PM, kir <***@gmail.com> wrote:
> On Wed, May 30, 2012 at 8:21 PM, Gruenke, Matt <***@tycoint.com> wrote:
>> Are you suggesting that thread::cancel() should have some special case
>> code to detect that the thread is calling io_service::run() and instead
>> call io_service::stop()?
>
> yes, exactly

oh sorry, not exactly.

If thread does io_service::run it must be interrupted same way as
io_service::stop would do, but without failing all pending operations.
Other threads could wake alongside with thread being interrupted.
Those must automatically restart (call ::run again). Then
boost::thread_interrupted exception must be thrown just as in other
interruption points.

real io_service::stop is called only when io_service destroyed.