[asio] accepting all connections in a seperate thread - how?
My server app needs to be "highly responsive". Ie. it shall handle connection attemps with a higher priority. Currently it has the following problem: if a handler function like the one below takes too much time, then unfortunately no new connections will be handled as long as the operation below lasts (ie. new connections have to wait). Which alternatives are there to solve this problem? (I was thinking of doing the accept in a seperate thread, but not sure how and whether that would work?) void session::handle_read(const boost::system::error_code& error, size_t bytes_transferred) { if (!error) { // PROBLEM HERE: // ... a lengthy job here blocks the whole app (ie. new connections have to wait) // ... which coding/design alternatives would solve this problem? boost::asio::async_write(socket_, boost::asio::buffer(data_, bytes_transferred), boost::bind(&session::handle_write, this, boost::asio::placeholders::error)); } else { delete this; } }
On Sun, Oct 10, 2010 at 4:57 PM, for-gmane <for-gmane@amitrader.com> wrote:
My server app needs to be "highly responsive". Ie. it shall handle connection attemps with a higher priority. Currently it has the following problem: if a handler function like the one below takes too much time, then unfortunately no new connections will be handled as long as the operation below lasts (ie. new connections have to wait). Which alternatives are there to solve this problem? (I was thinking of doing the accept in a seperate thread, but not sure how and whether that would work?)
void session::handle_read(const boost::system::error_code& error, size_t bytes_transferred) { if (!error) { // PROBLEM HERE: // ... a lengthy job here blocks the whole app (ie. new connections have to wait) // ... which coding/design alternatives would solve this problem?
boost::asio::async_write(socket_, boost::asio::buffer(data_, bytes_transferred), boost::bind(&session::handle_write, this, boost::asio::placeholders::error)); } else { delete this; } }
What I personally do (and not have had issues with *yet*) is create the standard io connecter, I feed it an initial thing that async waits on incoming connections, I think spawn a number of threads equal to the hardware_concurrency value just doing a run() in each of them on the io service, then in the main thread it just 'joins' them all waiting for them to die. In the initial thing that async waits on incoming connections, on an incoming connection it spawns a new async handler, tosses it away (I have it setup so when the socket dies, so does the handler, whether explicitly or implicitly) then immediately loops back and waits for another incoming connection asynchronously. The handlers wait on the socket for incoming data, send data, etc... Since the io service was 'run()' in multiple other threads, depending on the processor count, I get decently max usage of the cpu with my other worker data (which 'pause' themselves on occasion and setup a timer for one millisecond later to be called again and pick up where they left off until they complete their data, basically a cooperative multi-threading inside the pre-emptive multithreading model since I have a limited thread count).
On 2010-10-11 01:53, OvermindDL1 wrote:
On Sun, Oct 10, 2010 at 4:57 PM, for-gmane<for-gmane@amitrader.com> wrote:
My server app needs to be "highly responsive". Ie. it shall handle connection attemps with a higher priority. Currently it has the following problem: if a handler function like the one below takes too much time, then unfortunately no new connections will be handled as long as the operation below lasts (ie. new connections have to wait). Which alternatives are there to solve this problem? (I was thinking of doing the accept in a seperate thread, but not sure how and whether that would work?)
 void session::handle_read(const boost::system::error_code& error,   size_t bytes_transferred)  {   if (!error)   {    // PROBLEM HERE:    // ... a lengthy job here blocks the whole app (ie. new connections have to wait)    // ... which coding/design alternatives would solve this problem?
   boost::asio::async_write(socket_,      boost::asio::buffer(data_, bytes_transferred),      boost::bind(&session::handle_write, this,       boost::asio::placeholders::error));   }   else   {    delete this;   }  }
What I personally do (and not have had issues with *yet*) is create the standard io connecter, I feed it an initial thing that async waits on incoming connections, I think spawn a number of threads equal to the hardware_concurrency value just doing a run() in each of them on the io service, then in the main thread it just 'joins' them all waiting for them to die.
In the initial thing that async waits on incoming connections, on an incoming connection it spawns a new async handler, tosses it away (I have it setup so when the socket dies, so does the handler, whether explicitly or implicitly) then immediately loops back and waits for another incoming connection asynchronously.
The handlers wait on the socket for incoming data, send data, etc...
Since the io service was 'run()' in multiple other threads, depending on the processor count, I get decently max usage of the cpu with my other worker data (which 'pause' themselves on occasion and setup a timer for one millisecond later to be called again and pick up where they left off until they complete their data, basically a cooperative multi-threading inside the pre-emptive multithreading model since I have a limited thread count).
I think I've it similar to what you describe: a pool of threads (5) all to run of the io_service's run() method, and there is always an "outstanding" async_accept(), created even before start()ing the session for the current connection. Still, in the said scenario, none of the threads does handle the next connection attempt as long as the current session is doing a lengthy job. This is really funny. I think, asio allows the threads to pick up a job only after returning from the handler (ie. handle_read above). I'm not sure if that is not too restrictive. I even tried to do socket_.get_io_service().poll(); right before the lengthy operation, but this didn't help either.
On Sun, Oct 10, 2010 at 7:16 PM, for-gmane <for-gmane@amitrader.com> wrote:
On 2010-10-11 01:53, OvermindDL1 wrote:
On Sun, Oct 10, 2010 at 4:57 PM, for-gmane<for-gmane@amitrader.com> wrote:
My server app needs to be "highly responsive". Ie. it shall handle connection attemps with a higher priority. Currently it has the following problem: if a handler function like the one below takes too much time, then unfortunately no new connections will be handled as long as the operation below lasts (ie. new connections have to wait). Which alternatives are there to solve this problem? (I was thinking of doing the accept in a seperate thread, but not sure how and whether that would work?)
 void session::handle_read(const boost::system::error_code& error,   size_t bytes_transferred)  {   if (!error)   {    // PROBLEM HERE:    // ... a lengthy job here blocks the whole app (ie. new connections have to wait)    // ... which coding/design alternatives would solve this problem?
   boost::asio::async_write(socket_,      boost::asio::buffer(data_, bytes_transferred),      boost::bind(&session::handle_write, this,       boost::asio::placeholders::error));   }   else   {    delete this;   }  }
What I personally do (and not have had issues with *yet*) is create the standard io connecter, I feed it an initial thing that async waits on incoming connections, I think spawn a number of threads equal to the hardware_concurrency value just doing a run() in each of them on the io service, then in the main thread it just 'joins' them all waiting for them to die.
In the initial thing that async waits on incoming connections, on an incoming connection it spawns a new async handler, tosses it away (I have it setup so when the socket dies, so does the handler, whether explicitly or implicitly) then immediately loops back and waits for another incoming connection asynchronously.
The handlers wait on the socket for incoming data, send data, etc...
Since the io service was 'run()' in multiple other threads, depending on the processor count, I get decently max usage of the cpu with my other worker data (which 'pause' themselves on occasion and setup a timer for one millisecond later to be called again and pick up where they left off until they complete their data, basically a cooperative multi-threading inside the pre-emptive multithreading model since I have a limited thread count).
I think I've it similar to what you describe: a pool of threads (5) all to run of the io_service's run() method, and there is always an "outstanding" async_accept(), created even before start()ing the session for the current connection. Still, in the said scenario, none of the threads does handle the next connection attempt as long as the current session is doing a lengthy job. This is really funny. I think, asio allows the threads to pick up a job only after returning from the handler (ie. handle_read above). I'm not sure if that is not too restrictive. I even tried to do socket_.get_io_service().poll(); right before the lengthy operation, but this didn't help either.
Correct, only after returning from the handler for a given thread, that is why my primary loop is like: functionLoop { //have an incoming connection connectionHandler(socket); // this is just a class, it creates an async callback to call its operator() method, this then returns, I do other things, then do: socket->asyncWhateverIncomingConnectionMethod(*this); // then it returns and is called back in later, simple functors } How are yours structured?
On 2010-10-11 03:16, for-gmane wrote:
I think I've it similar to what you describe: a pool of threads (5) all to run of the io_service's run() method, and there is always an "outstanding" async_accept(), created even before start()ing the session for the current connection. Still, in the said scenario, none of the threads does handle the next connection attempt as long as the current session is doing a lengthy job. This is really funny. I think, asio allows the threads to pick up a job only after returning from the handler (ie. handle_read above). I'm not sure if that is not too restrictive. I even tried to do socket_.get_io_service().poll(); right before the lengthy operation, but this didn't help either.
AFAIK this is not the case; Asio doesn't do any dependency tracking of handlers, so new handlers should be executed immediately. Perhaps all your threads have run out of work (i.e., there's no work object attached to the io_service)? Have you tried posting your lengthy job to the task queue (io_service.post())? Regards, Rutger
On 2010-10-11 08:59, Rutger ter Borg wrote:
On 2010-10-11 03:16, for-gmane wrote:
I think I've it similar to what you describe: a pool of threads (5) all to run of the io_service's run() method, and there is always an "outstanding" async_accept(), created even before start()ing the session for the current connection. Still, in the said scenario, none of the threads does handle the next connection attempt as long as the current session is doing a lengthy job. This is really funny. I think, asio allows the threads to pick up a job only after returning from the handler (ie. handle_read above). I'm not sure if that is not too restrictive. I even tried to do socket_.get_io_service().poll(); right before the lengthy operation, but this didn't help either.
AFAIK this is not the case; Asio doesn't do any dependency tracking of handlers, so new handlers should be executed immediately. Perhaps all your threads have run out of work (i.e., there's no work object attached to the io_service)? Have you tried posting your lengthy job to the task queue (io_service.post())?
Indeed! Using io_service.post() solves the problem! Many thanks, very much appreciated!
participants (1)
-
for-gmane -
OvermindDL1 -
Rutger ter Borg