Hi boosters, I've implemented asynchronous file IO for linux in asio using aio_* functions. I created a class called basic_stream_file (and a typedef called stream_file) with almost the same interface as basic_stream_socket. But with read/write and async_read/async_write functions and an aio_demuxer_service. I writed it for the 0.3.2 version. For now to be able to use this new demuxer, is needed to create a demuxer of type: asio::basic_demuxer<asio::detail::aio_demuxer_service>. The implementation is attached. I didnt had time to test it very much, but it seems to be working okay. There are some problems that are related to aio_* in linux, since it isnt completely implemented in today's kernels and isnt implemented at all in 2.4.* kernels(there are patches however). And the aio_* functions only work really asynchronous for now if used with O_DIRECT and aligned buffers. But at least it could have a single interface for AIO with files in Windows and Linux. I'm planning to write asynchronous file I/O for windows with io completions ports too. IMO it would be interesting to be able to have a single demuxer in windows for file and socket AIO. I'm sending attached a file that uses these classes as an example. It must be linked with -laio and -lpthread and libaio must be installed. (It wasnt in my slackware 10.1, dont know about the default setup of other distributions). Any feedback will be appreciated. Thanks, Felipe. -- Felipe Magno de Almeida Developer from synergy and Computer Science student from State University of Campinas(UNICAMP). Unicamp: http://www.ic.unicamp.br Synergy: http://www.synergy.com.br "There is no dark side of the moon really. Matter of fact it's all dark."
Hi Felipe, --- Felipe Magno de Almeida <felipe.m.almeida@gmail.com> wrote:
I've implemented asynchronous file IO for linux in asio using aio_* functions. I created a class called basic_stream_file (and a typedef called stream_file) with almost the same interface as basic_stream_socket. But with read/write and async_read/async_write functions and an aio_demuxer_service. I writed it for the 0.3.2 version.
Nice! It didn't really require that much code either, which is impressive.
I didnt had time to test it very much, but it seems to be working okay. There are some problems that are related to aio_* in linux, since it isnt completely implemented in today's kernels and isnt implemented at all in 2.4.* kernels(there are patches however). And the aio_* functions only work really asynchronous for now if used with O_DIRECT and aligned buffers. But at least it could have a single interface for AIO with files in Windows and Linux.
Yeah, I can't really think of a good solution to the problem of correctly aligning the buffers. We may have to provide some sort of allocator object that is guaranteed to return buffers correctly aligned for the read and write operations.
Any feedback will be appreciated.
My only suggestion at this time is that rather than trying to reimplement all the demuxer code (for doing dispatch(), post() etc), just use the task_demuxer_service and implement the aio code as a task. A task must have the following interface: class Task { public: void reset(); // Reset in preparation for a new run() call. void run(); // Perform operations until interrupted. void interrupt(); // Stop an existing run loop. }; Note that the interrupt() function must be able to be called from a different thread to run(). You then use your demuxer like so: typedef basic_demuxer< detail::task_demuxer_service<detail::aio_task> > aio_demuxer; If you look at the select_reactor or epoll_reactor you will see that they implement the above Task concept, but they also support running themselves in a separate thread. This is one way to have aio for files and select_reactor/epoll_reactor for sockets coexist in the same demuxer object. Cheers, Chris
On 9/6/05, Christopher Kohlhoff <chris@kohlhoff.com> wrote:
Hi Felipe,
Hi,
[snip]
Yeah, I can't really think of a good solution to the problem of correctly aligning the buffers. We may have to provide some sort of allocator object that is guaranteed to return buffers correctly aligned for the read and write operations.
The solution can be that, if someone needs real asynchronous AIO for files in linux, that it patch its kernel, or that it takes care of using O_DIRECT flag and aligning the buffers manually before using it with asio. Since, it can be that kernel 2.6 gets AIO soon(I dont have any idea when it will happen), and then we would have useless code and interface in the library... Well, just a though, I'm still not sure...
Any feedback will be appreciated.
My only suggestion at this time is that rather than trying to reimplement all the demuxer code (for doing dispatch(), post() etc), just use the task_demuxer_service and implement the aio code as a task. A task must have the following interface:
class Task { public: void reset(); // Reset in preparation for a new run() call. void run(); // Perform operations until interrupted. void interrupt(); // Stop an existing run loop. };
Note that the interrupt() function must be able to be called from a different thread to run().
You then use your demuxer like so:
typedef basic_demuxer< detail::task_demuxer_service<detail::aio_task> > aio_demuxer;
I think I missed this part of the code, now that I looked better I'm certain this is the best way to do it. Could you give me a rationale of the Task concept name?
If you look at the select_reactor or epoll_reactor you will see that they implement the above Task concept, but they also support running themselves in a separate thread. This is one way to have aio for files and select_reactor/epoll_reactor for sockets coexist in the same demuxer object.
It is in the top of my TODO list. Next in this list is a little more though about basic_stream_file's constructor.
Cheers, Chris
Thanks very much, -- Felipe Magno de Almeida Developer from synergy and Computer Science student from State University of Campinas(UNICAMP). Unicamp: http://www.ic.unicamp.br Synergy: http://www.synergy.com.br "There is no dark side of the moon really. Matter of fact it's all dark."
On 9/7/05, Felipe Magno de Almeida <felipe.m.almeida@gmail.com> wrote:
On 9/6/05, Christopher Kohlhoff <chris@kohlhoff.com> wrote:
My only suggestion at this time is that rather than trying to reimplement all the demuxer code (for doing dispatch(), post() etc), just use the task_demuxer_service and implement the aio code as a task. A task must have the following interface:
class Task { public: void reset(); // Reset in preparation for a new run() call. void run(); // Perform operations until interrupted. void interrupt(); // Stop an existing run loop. };
Note that the interrupt() function must be able to be called from a different thread to run().
You then use your demuxer like so:
typedef basic_demuxer< detail::task_demuxer_service<detail::aio_task> > aio_demuxer;
I think I missed this part of the code, now that I looked better I'm certain this is the best way to do it. Could you give me a rationale of the Task concept name?
Perhaps the active object (ACE_Task) in ACE? The asio Task doesn't have a message queue like the ACE one, but it is otherwise similar. -- Caleb Epstein caleb dot epstein at gmail dot com
Hi Caleb, Felipe, Sorry for delayed reply on this one... --- Caleb Epstein <caleb.epstein@gmail.com> wrote:
On 9/7/05, Felipe Magno de Almeida <felipe.m.almeida@gmail.com> wrote:
Could you give me a rationale of the Task concept name?
Perhaps the active object (ACE_Task) in ACE? The asio Task doesn't have a message queue like the ACE one, but it is otherwise similar.
It's similar but not quite the same, since an ACE_Task is normally used as a separate thread by itself (since it provides a function for starting a thread). The Task concept in asio on the other hand is run in a thread, but does not own that thread itself (and in the case of the task_demuxer_service the task is interrupted to perform other work like dispatching completion handlers from the same thread). I chose the name Task only because it was a nice short name that meant "something to be done", where that thing could interrupted and set aside as necessary. Cheers, Chris
participants (5)
-
Caleb Epstein -
Christopher Kohlhoff -
Felipe Magno de Almeida