[boost thread rework] New version uploaded to Boost Vault
Hi to all, I've just finished implementing the lockable objects of Kevlin's proposal based on Boost mutexes and conditions. Since boost mutex operations are private and only available with locks, I haven't found a way to implement Kevlin's interface over boost threads without using some very nasty tricks. If someone has a better idea, let me know. The implementation nearly fills all the proposal of the documentation, so I will wait for new elements so that I can continue with this project. This elements are present in the current version: -> threader and joiner classes -> lockable, try_lockable, condition_lockable -> locker, try_locker -> locking_ptr There are open issues in the paper like timeout handling and new possible lockers, (for example, for conditional_lockable objects). As soon as I get more information I will continue developing it. Regards, Ion
Ion Gaztañaga <igaztanaga@gmail.com> writes:
Hi to all,
I've just finished implementing the lockable objects of Kevlin's proposal based on Boost mutexes and conditions. Since boost mutex operations are private and only available with locks, I haven't found a way to implement Kevlin's interface over boost threads without using some very nasty tricks. If someone has a better idea, let me know.
The implementation nearly fills all the proposal of the documentation, so I will wait for new elements so that I can continue with this project. This elements are present in the current version:
-> threader and joiner classes -> lockable, try_lockable, condition_lockable -> locker, try_locker -> locking_ptr
There are open issues in the paper like timeout handling and new possible lockers, (for example, for conditional_lockable objects). As soon as I get more information I will continue developing it.
I don't want to be a buzz-kill and I know it's not glamorous, but I'm really concerned about the health of the existing Boost.Thread primitives. IIUC, none of these cool high-level components you're working on can act as full replacements for the existing stuff. In particular, for example, the Boost.Thread documentation needs a lot of work. The code, apparently, needs a rewrite in order to get it to use the Boost license, but that's probably too much to hope for right now. I'd really like to know that the existing library is being actively and conscientiously maintained before we move on to bigger and better things. -- Dave Abrahams Boost Consulting www.boost-consulting.com
On 01/09/05, David Abrahams <dave@boost-consulting.com> wrote: Ion Gaztañaga <igaztanaga@gmail.com> writes:
I don't want to be a buzz-kill and I know it's not glamorous, but I'm really concerned about the health of the existing Boost.Thread primitives. IIUC, none of these cool high-level components you're working on can act as full replacements for the existing stuff.
In particular, for example, the Boost.Thread documentation needs a lot of work. The code, apparently, needs a rewrite in order to get it to use the Boost license, but that's probably too much to hope for right now.
I'd really like to know that the existing library is being actively and conscientiously maintained before we move on to bigger and better things.
Perhaps not too much to ask for... I tend to agree with David's sentiment. I'd be happy to start a simplification and restructure of the basic primitives. The code should be restructured into platform specific implementations, like boost has elsewhere (such as the atomic inc for shared_prt), as the current code I find a little tricky to read. With such a restructure the various platforms could have different maintainers to eliminate a bit of the difficulty of trying to support such a cross platform library. I could pull together a basic posix-based implementation which could be used on win32 with the posix32 lib as well, though a native win32 should happen as most people would find the posix32 layer unacceptable. Perhaps the current implementation can be refactored to do this, but starting from a clean slate so that the work can be licensed under the boost license might be best. Level 0 - basic atomic ops, fencing, thread, mutex (normal, recursive, rw), condition - to be replaced and updated with the work happening elsewhere by Lea, Boehm etal. This should also replace the atomic ops used by shared_ptr over time. Should be in a style that suits generic programming via a consistent interface which is currently lacking. Level 1 - futures, threadables, message queues, etc, primitives along the direction of Henney's work. Level 2 - Framework abstractions to architect single process, multi-process, multi-computer workflows. Needs boost::net / comms to reach its potential. Any thoughts? matt. matt@zomojo.com
Matt Hurd <matt.hurd@gmail.com> writes:
I tend to agree with David's sentiment. I'd be happy to start a simplification and restructure of the basic primitives. The code should be restructured into platform specific implementations, like boost has elsewhere (such as the atomic inc for shared_prt), as the current code I find a little tricky to read.
Please, whatever you can do! But please do a fresh rewrite. That doesn't mean you can't look at the old code; you just have to type new characters ;-)
With such a restructure the various platforms could have different maintainers to eliminate a bit of the difficulty of trying to support such a cross platform library.
Makes sense.
I could pull together a basic posix-based implementation which could be used on win32 with the posix32 lib as well, though a native win32 should happen as most people would find the posix32 layer unacceptable. Perhaps the current implementation can be refactored to do this, but starting from a clean slate so that the work can be licensed under the boost license might be best.
Agree.
Level 0 - basic atomic ops, fencing, thread, mutex (normal, recursive, rw), condition - to be replaced and updated with the work happening elsewhere by Lea, Boehm etal. This should also replace the atomic ops used by shared_ptr over time. Should be in a style that suits generic programming via a consistent interface which is currently lacking.
Level 1 - futures, threadables, message queues, etc, primitives along the direction of Henney's work.
Level 2 - Framework abstractions to architect single process, multi-process, multi-computer workflows. Needs boost::net / comms to reach its potential.
Any thoughts?
Not familiar enough with the library internals to comment. -- Dave Abrahams Boost Consulting www.boost-consulting.com
On 01/09/05, David Abrahams <dave@boost-consulting.com> wrote: Matt Hurd <matt.hurd@gmail.com> writes:
Please, whatever you can do! But please do a fresh rewrite. That doesn't mean you can't look at the old code; you just have to type new characters ;-)
OK, I'll start afresh. The code should be the easy part, the documentation will be the main body of work I suspect.
With such a restructure the various platforms could have different maintainers to eliminate a bit of the difficulty of trying to support such a cross platform library.
Makes sense.
I could pull together a basic posix-based implementation which could be used on win32 with the posix32 lib as well, though a native win32 should happen as most people would find the posix32 layer unacceptable. Perhaps the current implementation can be refactored to do this, but starting from a clean slate so that the work can be licensed under the boost license might be best.
Agree.
OK.
Level 0 - basic atomic ops, fencing, thread, mutex (normal, recursive, rw), condition - to be replaced and updated with the work happening elsewhere by Lea, Boehm etal. This should also replace the atomic ops used by shared_ptr over time. Should be in a style that suits generic programming via a consistent interface which is currently lacking.
Level 1 - futures, threadables, message queues, etc, primitives along the direction of Henney's work.
Level 2 - Framework abstractions to architect single process, multi-process, multi-computer workflows. Needs boost::net / comms to reach its potential.
Any thoughts?
Not familiar enough with the library internals to comment.
A solid level 0 that can quickly be adapted to the direction of the ISO C++ threading push and provide a foundation for the level 1 stuff wil be goal I guess. I expect there to be a level -1 that would support the level of control that 95% of us don't need with a couple of dozen synch primitives for various flavours of acquire release and the like. That is not for now. I'll add to the wiki or put up a page here at work to support the effort by first thing next week. Regards, Matt. matt@zomojo.com
Matt Hurd schrieb:
I'd really like to know that the existing library is being actively and conscientiously maintained before we move on to bigger and better things.
Perhaps not too much to ask for...
I tend to agree with David's sentiment. I'd be happy to start a simplification and restructure of the basic primitives. The code should be restructured into platform specific implementations, like boost has elsewhere (such as the atomic inc for shared_prt), as the current code I find a little tricky to read.
With such a restructure the various platforms could have different maintainers to eliminate a bit of the difficulty of trying to support such a cross platform library.
This is the second time I see this happen :-( Bill Kempf, the original author of the library silently disappeared. The reason always was unclear to me. It looks to me as if he was disappointed that the C++ comitee did not accept anything of the threading work so far. Please correct me if this are wrong impressions. At least I learned this from digging through the muddy waters of the mailing list archive. Next Michael Glassford appeared, trying hard to improve on the library. I don't know what happened. I don't know whether he underestimated the task or he is beeing teared apart by something different. At a minimum I would like to hear what he has to say about the new plans. (I also BCC this to him directly.) In the end this all led to a major starvation on the threading issue. Also there are some areas that cannot be addressed with library code alone (I think): As Andrej Alexandrescu et. al. were poining out some semantic changes to the compiler are in order to provide a true multihreaded architecture. (Atomicity, memory model,...) On the other side there is a need for multithreading that can do with current compilers. (And the current library is doing quite well.)
I could pull together a basic posix-based implementation which could be used on win32 with the posix32 lib as well, though a native win32 should happen as most people would find the posix32 layer unacceptable.
I would rather like to see the current interface kept as much as possible. Did anyone yet try to find out how deeply the boost::thread is currently wired into the boost lib? I think it would be a good staring point to find this out before any interface changes are beeing introduced, or else we will end with the need to support two possibly incompatible threading libraries. And keeping the interfaces would not rise copyrigth issues would they?
Perhaps the current implementation can be refactored to do this, but starting from a clean slate so that the work can be licensed under the boost license might be best.
Refactoring looks attractive to me, but keep the interfaces.
Level 0 - basic atomic ops, fencing, thread, mutex (normal, recursive, rw), condition - to be replaced and updated with the work happening elsewhere by Lea, Boehm etal. This should also replace the atomic ops used by shared_ptr over time. Should be in a style that suits generic programming via a consistent interface which is currently lacking.
Level 1 - futures, threadables, message queues, etc, primitives along the direction of Henney's work.
Level 2 - Framework abstractions to architect single process, multi-process, multi-computer workflows. Needs boost::net / comms to reach its potential.
I am missing TLS from this list.
Any thoughts?
I like the idea of splitting the lib to maintainer/platform a lot. But we would need to agree on a coordinated process for defining the interfaces. Any suggestions? At the same time I again announce my willingness to volunteer for the win32 part. I already gained some insight when working on the TLS implementation for windows. I also attempted a clean rewrite of TLS (using the same interfaces) for the windows platform, that resulted in a aprox 4times speed improvement. Unfortunately I never had a chance to discuss this on the list. Then I suggested a so called thread alerter which can also solve most of the practical thread cancellation issues. (It is not true cancellation though.) Considering the constant requests for addition of thread cancellation, surprisingly there was almost no feedback on this at all Regards Roland
On 01/09/05, Roland Schwarz <roland.schwarz@chello.at> wrote:
Matt Hurd schrieb: <snip> This is the second time I see this happen :-(
Michael Glassford is still the maintainer. He has commented that he is fairly busy and I think supporting the detail that is required on the various platforms is daunting. It has been a big job that Michael has undertaken to update the documentation and get thread from where it was to where it is now. Kudos to Michael otherwise perhaps boost::thread wouldn't be here at all any more. <snip> It would be good to hear Michael's thoughts as he has especially done a lot of work on the documentation and this is perhaps the biggest aspect of the lib.
In the end this all led to a major starvation on the threading issue.
Also there are some areas that cannot be addressed with library code alone (I think): As Andrej Alexandrescu et. al. were poining out some semantic changes to the compiler are in order to provide a true multihreaded architecture. (Atomicity, memory model,...)
On the other side there is a need for multithreading that can do with current compilers. (And the current library is doing quite well.)
Yes. There needs to be agreement at some stage on the memory model with respect to visibility of operations when threads are in play, agreement on the definition of a data race, and the kind of optimizations that a compiler may do around such code points. This tends to be very platform specific presently and the problem is that often there is a sneaky less well know platform specific memory model aspect that can benefit a particular critical case. For example, from memory, Sparc supports more than a dozen variations on a theme of basic memory synch operations. We need to avoid that detail and leave it to the ISO C++ concurrency guys otherwise we will not make progress. Plenty of low hanging fruit to grab first. I think we can push ahead with basic atomic ops (inc, dec, sub, add, swap, cas?), simple fencing (load, store, full). I think I've got that covered for x86 on linux and win32 and hopefully others will contribute code for their platforms. Not sure about the fencing as at least a generic mutex implementation can make atomic ops portable whilst we wait for platform specific implementations. Maybe a generic lock/unlock of mutex can be an ugly (very ugly) proxy for a full fence ( as if not, we are perhaps in trouble with visibility on that platform anyway) but I expect that this may not hold and the only viable portable implementation will be a static assert ;-) The ISO C++ threading push will come up with more detailed and better thought out proposals that we should listen carefully to and adapt to. But that is somewhat orthogonal to getting boost::thread rewritten.
I could pull together a basic posix-based implementation which could be used on win32 with the posix32 lib as well, though a native win32 should happen as most people would find the posix32 layer unacceptable.
I would rather like to see the current interface kept as much as possible. Did anyone yet try to find out how deeply the boost::thread is currently wired into the boost lib?
Good point. I'll have a look at where a mutex is currently used. I don't think threads and thread pools are used too much in the libs.
I think it would be a good staring point to find this out before any interface changes are beeing introduced, or else we will end with the need to support two possibly incompatible threading libraries. And keeping the interfaces would not rise copyrigth issues would they?
Perhaps the current implementation can be refactored to do this, but starting from a clean slate so that the work can be licensed under the boost license might be best.
Refactoring looks attractive to me, but keep the interfaces.
Perhaps some changes if agreed to. Similar basic thread ops and scoped locking of mutexes should be consistent but now rather than later is the time to make changes if there are any good arguments. One good thing about avoiding too much change is at least it will get done as there many nice ideas (the transfer locking thingo discussed sometime ago springs to mind) that are perhaps best explored _after_ the code is restructured. My current project could do with this lib so I would like to get it done without too much delay.
I am missing TLS from this list.
Yep. TLS should be in level 0 I'd think if we can squeeze it in without too much effort. I think some simple thread priority stuff, along with a platform specific thread id, can slip in too without much ado as well. These seem to be FAQs.
Any thoughts?
I like the idea of splitting the lib to maintainer/platform a lot. But we would need to agree on a coordinated process for defining the interfaces. Any suggestions?
I'll organise a central point by early next week and start a doc if agreed. I'd prefer something like an open office doc to a wiki. Changes can be annotated, filtered back and accepted and re-published to the site as a doc and as html. I'm a bit of a dinosaur w.r.t. to this new fangled wiki stuff...
At the same time I again announce my willingness to volunteer for the win32 part.
Super. Most of my commercial experience has been with win32 but I rarely use it now.
I already gained some insight when working on the TLS implementation for windows. I also attempted a clean rewrite of TLS (using the same interfaces) for the windows platform, that resulted in a aprox 4times speed improvement. Unfortunately I never had a chance to discuss this on the list.
Then I suggested a so called thread alerter which can also solve most of the practical thread cancellation issues. (It is not true cancellation though.) Considering the constant requests for addition of thread cancellation, surprisingly there was almost no feedback on this at all
It is a good idea but probably slots into level 1 which I will avoiding to start with so I can focus on the grunt work of recoding William's and Michael's fine work. Maybe some level 1 stuff can go on concurrently with level 0. I hope level 0 will be straight forward enough. The main part that scares me is the trickiness of the static linking win32 issues. The linux posix implementation will cover many bases... Candidate platform maintainers: Roland Schwarz: win32 x86 Michael Glassford: ?? Ion Gaztañaga: ?? Matt Hurd: linux x86_64 Any volunteers for specific platforms? No need for just one per platform. Regards, Matt.
Matt Hurd schrieb:
On 01/09/05, Roland Schwarz <roland.schwarz@chello.at> wrote:
Matt Hurd schrieb:
<-->8-- snipped -->8-- >
I would rather like to see the current interface kept as much as possible. Did anyone yet try to find out how deeply the boost::thread is currently wired into the boost lib?
Good point. I'll have a look at where a mutex is currently used. I don't think threads and thread pools are used too much in the libs.
<-->8-- snipped -->8-- >
I am missing TLS from this list.
Yep. TLS should be in level 0 I'd think if we can squeeze it in without too much effort. I think some simple thread priority stuff, along with a platform specific thread id, can slip in too without much ado as well. These seem to be FAQs.
Not sure about this. I think the TLS could very well reside in a separate unit. It is almost orthogonal to threads. Of course there is a minor dependancy, but that could be easily solved by introduction of an at_exit_thread mechanism. (Which by the way is already used under the hoods.) On the other hand the concept of thread id's can be based on an extended version of TLS. I already submitted a draft of this that can be found in my thread alert submission. So in some sense there might be some dependancy from threads upon TLS.
I'll organise a central point by early next week and start a doc if agreed. I'd prefer something like an open office doc to a wiki. Changes can be annotated, filtered back and accepted and re-published to the site as a doc and as html. I'm a bit of a dinosaur w.r.t. to this new fangled wiki stuff...
Fine! But I am not sure if switching to open office will set some additional barriers for others. Why not simply using the infrastucture that is already there? There is already a wiki installed, and there is also a thread section in it. Could we give it a try? Also introducing a new method might only artificially increase the ever present unavoidable entropy. I think we rather should put energy in it to decrease it (at least locally ;-) <-->8-- snipped -->8-- >
The main part that scares me is the trickiness of the static linking win32 issues.
I am in a good position to deal with these issues, since it was me who contributed the static linking code parts.
The linux posix implementation will cover many bases...
Given. But what I see as a first major challenge is how to organize the library to split out the platform dependant parts into unique units. The current method of #ifdef ' ing is not the very best I think. Perhaps we should set up a split directory structure instead? But then I see two extremes: 1) A common header which is the only point where platform specific subheaders are ifdeffed into play. Building the lib is a separate issue and possibly could be solved by tweaking the bjam accordingly. 2) A common set of headers that defines the interface only and leave implementation to the platform files. However this seems prohibitive since this would forbid for platform specific inline optimizations. Point 1) is at risk, that interfaces start to diverge and 2) still has the ugly ifdefs. Any thoughts? Regards, Roland
From: Roland Schwarz <roland.schwarz@chello.at>
Perhaps we should set up a split directory structure instead?
But then I see two extremes: 1) A common header which is the only point where platform specific subheaders are ifdeffed into play. Building the lib is a separate issue and possibly could be solved by tweaking the bjam accordingly.
2) A common set of headers that defines the interface only and leave implementation to the platform files. However this seems prohibitive since this would forbid for platform specific inline optimizations.
Point 1) is at risk, that interfaces start to diverge and 2) still has the ugly ifdefs.
Breaking the platform-specific code into separate files keeps separate things separate and makes everything more readable. That seems like the appropriate course. However, doing so doesn't mean the interfaces need to diverge. I suggest this variation: 3) A common header or set of common headers that declare(s) the interfaces plus #ifdefs to include the appropriate platform-specific header(s). Thus, when including a.hpp, via conditional compilation, you may also get a_1.hpp, a_2.hpp, or a_3.hpp. Whether there are corresponding files a_1.cpp, a_2.cpp, a_3.cpp, or even a.cpp, is based upon need. The use of platform-specific headers allows for inline code, while the common header(s) allow(s) for a simply to grok declaration of the interfaces. -- Rob Stewart stewart@sig.com Software Engineer http://www.sig.com Susquehanna International Group, LLP using std::disclaimer;
On 02/09/05, Rob Stewart <stewart@sig.com> wrote:
From: Roland Schwarz <roland.schwarz@chello.at>
Perhaps we should set up a split directory structure instead?
That's the plan. Also give some clearer responsibilities for platform maintainers.
But then I see two extremes: 1) A common header which is the only point where platform specific
<snip>
2) A common set of headers that defines the interface only and leave implementation to the platform files. However this seems prohibitive since this would forbid for platform specific inline optimizations.
<snip>
3) A common header or set of common headers that declare(s) the interfaces plus #ifdefs to include the appropriate platform-specific header(s). Thus, when including a.hpp, via conditional compilation, you may also get a_1.hpp, a_2.hpp, or a_3.hpp. Whether there are corresponding files a_1.cpp, a_2.cpp, a_3.cpp, or even a.cpp, is based upon need.
2) doesn't fly due to the inlining. Something along the lines of 1 and 3. I'm not worried too much about interface inconsistencies as such an occurance will just highlight the failure of the test cases. The larger issue will be that many platforms will just be a posix implementation variation perhaps with only minor changes. Perhaps it will make sense to have a generic posix implementation that is #ifdef'd per platform. This will be referred to as the default implementation. We can make a call when the #ifdef'ing detracts too much and break such an implementation out to its own. matt. matt@zomojo.com
On 01/09/05, Roland Schwarz <roland.schwarz@chello.at> wrote:
Matt Hurd schrieb:
Yep. TLS should be in level 0 I'd think if we can squeeze it in without too much effort. I think some simple thread priority stuff, along with a platform specific thread id, can slip in too without much ado as well. These seem to be FAQs.
Not sure about this. I think the TLS could very well reside in a separate unit. It is almost orthogonal to threads. Of course there is a minor dependancy, but that could be easily solved by introduction of an at_exit_thread mechanism. (Which by the way is already used under the hoods.) On the other hand the concept of thread id's can be based on an extended version of TLS. I already submitted a draft of this that can be found in my thread alert submission. So in some sense there might be some dependancy from threads upon TLS.
We'll see how we travel. Just getting the current interface rewritten will be a start. I do hope we can take that momentum and tackle the other needs quickly.
I'll organise a central point by early next week and start a doc if agreed. I'd prefer something like an open office doc to a wiki. Changes can be annotated, filtered back and accepted and re-published to the site as a doc and as html. I'm a bit of a dinosaur w.r.t. to this new fangled wiki stuff...
Fine! But I am not sure if switching to open office will set some additional barriers for others. Why not simply using the infrastucture that is already there? There is already a wiki installed, and there is also a thread section in it. Could we give it a try? Also introducing a new method might only artificially increase the ever present unavoidable entropy. I think we rather should put energy in it to decrease it (at least locally ;-)
I'll check out the wiki.
<-->8-- snipped -->8-- >
The main part that scares me is the trickiness of the static linking win32 issues.
I am in a good position to deal with these issues, since it was me who contributed the static linking code parts.
Super!
But what I see as a first major challenge is how to organize the library to split <snip> agree to the split directories, see earlier response to later post.
matt.
Candidate platform maintainers:
Roland Schwarz: win32 x86 Michael Glassford: ?? Ion Gaztañaga: ?? Matt Hurd: linux x86_64
Any volunteers for specific platforms? No need for just one per platform.
Sorry to tell that level 0 is far from my knowledge, but I'm willing to help in any other aspect. Regards, Ion
Matt Hurd <matt.hurd@gmail.com> writes: Roland wrote:
I would rather like to see the current interface kept as much as possible. Did anyone yet try to find out how deeply the boost::thread is currently wired into the boost lib?
Good point. I'll have a look at where a mutex is currently used. I don't think threads and thread pools are used too much in the libs.
I have to agree with Roland. I hope you're not thinking of major interface changes. IMO the interface is quite well thought-out for the domain and level of abstraction it covers. The problems lie elsewhere. -- Dave Abrahams Boost Consulting www.boost-consulting.com
On 02/09/05, David Abrahams <dave@boost-consulting.com> wrote: Matt Hurd <matt.hurd@gmail.com> writes:
Roland wrote:
I would rather like to see the current interface kept as much as possible. Did anyone yet try to find out how deeply the boost::thread is currently wired into the boost lib?
Good point. I'll have a look at where a mutex is currently used. I don't think threads and thread pools are used too much in the libs.
I have to agree with Roland. I hope you're not thinking of major interface changes. IMO the interface is quite well thought-out for the domain and level of abstraction it covers. The problems lie elsewhere.
Just a rewrite to start. There are some problems with the interface I see that prevent clean generic use, but the existing stuff can be re-wrapped, as I do in my current code base, to provide such an interface so it is not worth delaying to consider. Let's just get it restructured first. Much of the new stuff people have planned will fit into level 1. Level 1 requirements will drive interface changes if need be I suspect. Additional basic ops such as atomics and fencing will slot into level 0 pretty easily given a platform specific approach but that is orthogonal to the rewrite. I hope we can tackle this pretty quickly after the restructure as this will complete 90% of the services extended frameworks will need. matt.
Matt Hurd <matt.hurd@gmail.com> writes:
Candidate platform maintainers:
Roland Schwarz: win32 x86 Michael Glassford: ?? Ion Gaztañaga: ?? Matt Hurd: linux x86_64
Any volunteers for specific platforms? No need for just one per platform.
I'm happy to work with Roland on the win32 port; I've looked at lots of the issues relating to thread primitives on Windows before. Anthony -- Anthony Williams Software Developer
On 02/09/05, Anthony Williams <anthony_w.geo@yahoo.com> wrote:
Any volunteers for specific platforms? No need for just one per platform.
I'm happy to work with Roland on the win32 port; I've looked at lots of the issues relating to thread primitives on Windows before.
Anthony -- Anthony Williams Software Developer
Super! Platform maintainers: Roland Schwarz: win32 x86 Anthony Williams: win32 x86 Matt Hurd: linux x86_64 matt.
Matt Hurd <matt.hurd@gmail.com> writes:
On 02/09/05, Anthony Williams <anthony_w.geo@yahoo.com> wrote:
Any volunteers for specific platforms? No need for just one per platform.
I'm happy to work with Roland on the win32 port; I've looked at lots of the issues relating to thread primitives on Windows before.
Super!
Right-oh. Here's two implementations of call_once (attached), for starters. once.hpp uses a Semaphore, whereas once_mutex.hpp uses a mutex, the same as the existing boost::thread implementation. Running the test program, the semaphore version gives a timing of 16s compiled with gcc-mingw-4.0.1, and 22s compiled with MSVC 7.1, on my machine. The mutex version gives timings of 23s (gcc) and 29s (MSVC), so the semaphore version is clearly faster. The mutex version allows for comparison of the once_flag instance against BOOST_ONCE_INIT, as implicitly suggested by the docs, whereas the semaphore version doesn't allow this, as BOOST_ONCE_INIT is an aggregate initializer. Also, the semaphore version runs the function once, even if it throws an exception. The mutex version runs the function repeatedly if it throws an exception. The docs say that the function isn't allowed to throw, so this is a moot point. I'd rather go with the semaphore version, but if others prefer the mutex version, I'll go with that. Both versions would need touching-up to handle wide-char-only platforms (winCE?), and platforms without stringstream. Anthony -- Anthony Williams Software Developer
Anthony Williams wrote:
Also, the semaphore version runs the function once, even if it throws an exception. The mutex version runs the function repeatedly if it throws an exception. The docs say that the function isn't allowed to throw, so this is a moot point.
I like the signature of call_once: template<typename Function> void call_once(Function f, once_flag& flag) but you need to fix the above behavior. The function is considered "run" only when it returns without an exception. So your semaphore version runs it zero times when an exception is thrown.
Peter Dimov wrote: [...]
I like the signature of call_once:
template<typename Function> void call_once(Function f, once_flag& flag)
Only as long as you stick to void Functions. Or am I just missing something? regards, alexander.
Alexander Terekhov wrote:
Peter Dimov wrote: [...]
I like the signature of call_once:
template<typename Function> void call_once(Function f, once_flag& flag)
Only as long as you stick to void Functions. Or am I just missing something?
Well, it's better than void(*)() and has enough expressive power to emulate the rest. The "typename result_of<Function()>::type" version is slightly harder to specify and implement. Probably not much, though.
Peter Dimov wrote:
Alexander Terekhov wrote:
Peter Dimov wrote: [...]
I like the signature of call_once:
template<typename Function> void call_once(Function f, once_flag& flag)
Only as long as you stick to void Functions. Or am I just missing something?
Well, it's better than void(*)() and has enough expressive power to emulate the rest.
The "typename result_of<Function()>::type" version is slightly harder to specify and implement. Probably not much, though.
On second thought, a non-void return doesn't make sense; what could it return when call_once is called a second time?
Alexander Terekhov wrote:
Peter Dimov wrote:
On second thought, a non-void return doesn't make sense; what could it return when call_once is called a second time?
The same thing as the first time.
Things like auto_ptr aside, you'd need to templatize the once_flag on the return type then, since the first time value needs to be stored somewhere. Is this worth the hassle?
Peter Dimov wrote: [...]
Things like auto_ptr aside, you'd need to templatize the once_flag on the return type then, since the first time value needs to be stored somewhere. Is this worth the hassle?
I think yes. regards, alexander.
Alexander Terekhov <terekhov@web.de> writes:
Peter Dimov wrote: [...]
Things like auto_ptr aside, you'd need to templatize the once_flag on the return type then, since the first time value needs to be stored somewhere. Is this worth the hassle?
I think yes.
How about the attached version? The result is stored on the heap, which adds some performance cost, but the once_flag itself doesn't need templating. I haven't yet adjusted it to avoid this heap allocation for void functions. It does mean that the initializer is now an aggregate initializer rather than a constant, as for my original Semaphore-based version. Anthony -- Anthony Williams Software Developer
Things like auto_ptr aside, you'd need to templatize the once_flag on the return type then, since the first time value needs to be stored somewhere. Is this worth the hassle?
I think yes.
How about the attached version? The result is stored on the heap, which adds some performance cost, but the once_flag itself doesn't need templating. I haven't yet adjusted it to avoid this heap allocation for void functions.
It does mean that the initializer is now an aggregate initializer rather than a constant, as for my original Semaphore-based version.
Interesting, but don't you end up with the classic singleton problem of "when do I delete this object?", doesn't this implementation risk deleting the object before the last call to the function? There may be workarounds, but I'm not convinced at the moment that they are worth the hassle, are we convinced that having a return value is really that important? I also have a slightly reworked version of call_once for you to consider, it's basically the same as your last version, but does away with the need to use stringstreams when creating the mutex name - this cuts down on the memory allocation calls, which should be a big win the first time through. Regards, John.
"John Maddock" <john@johnmaddock.co.uk> writes:
Things like auto_ptr aside, you'd need to templatize the once_flag on the return type then, since the first time value needs to be stored somewhere. Is this worth the hassle?
I think yes.
How about the attached version? The result is stored on the heap, which adds some performance cost, but the once_flag itself doesn't need templating. I haven't yet adjusted it to avoid this heap allocation for void functions.
It does mean that the initializer is now an aggregate initializer rather than a constant, as for my original Semaphore-based version.
Interesting, but don't you end up with the classic singleton problem of "when do I delete this object?", doesn't this implementation risk deleting the object before the last call to the function? There may be workarounds, but I'm not convinced at the moment that they are worth the hassle, are we convinced that having a return value is really that important?
Hmm. This implementation deletes the return value when the once_flag is destroyed, relying on the compiler to do this at an appropriate time. Of course, if the once_flag is a static, and the compiler chooses a poor order of static destruction, such that a destructor called later needs the once_flag, then you have a problem. The alternative is to not destroy the return value, which doesn't strike me as a good option. Maybe this *is* too much hassle.
I also have a slightly reworked version of call_once for you to consider, it's basically the same as your last version, but does away with the need to use stringstreams when creating the mutex name - this cuts down on the memory allocation calls, which should be a big win the first time through.
Thanks. I had meant to do something like this, but figured I'd leave it for later. Anthony -- Anthony Williams Software Developer
"Peter Dimov" <pdimov@mmltd.net> writes:
Anthony Williams wrote:
Also, the semaphore version runs the function once, even if it throws an exception. The mutex version runs the function repeatedly if it throws an exception. The docs say that the function isn't allowed to throw, so this is a moot point.
I like the signature of call_once:
template<typename Function> void call_once(Function f, once_flag& flag)
but you need to fix the above behavior. The function is considered "run" only when it returns without an exception. So your semaphore version runs it zero times when an exception is thrown.
void f() { throw "something"; } int main() { try { f(); } catch(...){} } How many times has f() been *called*? One, or none? I find your concept of a function only having run if it exits without an exception odd. In my mind call_once means just that --- call the function once, not let it run without an exception once. The docs (http://www.boost.org/regression-logs/cs-win32_metacomm/doc/html/call_once.ht...) say that "the function func shall not throw exceptions", so handling it this way is certainly acceptable, IMO. Anthony -- Anthony Williams Software Developer
From: "Anthony Williams"
"Peter Dimov" <pdimov@mmltd.net> writes:
Anthony Williams wrote:
Also, the semaphore version runs the function once, even if it throws an exception. The mutex version runs the function repeatedly if it throws an exception. The docs say that the function isn't allowed to throw, so this is a moot point.
I like the signature of call_once:
template<typename Function> void call_once(Function f, once_flag& flag)
but you need to fix the above behavior. The function is considered "run" only when it returns without an exception. So your semaphore version runs it zero times when an exception is thrown.
void f() { throw "something"; }
int main() { try { f(); } catch(...){} }
How many times has f() been *called*? One, or none? I find your concept of a function only having run if it exits without an exception odd.
It has been called once, but it didn't run to completion, and hence, its postconditions haven't been satisfied. Therefore, if a subsequent call_once returns successfully, as if nothing happened, the code will proceed under the assumption that the postconditions are met, and will break in various ways. Local statics retry construction on exception for the same reason.
Anthony Williams wrote: [...]
once.hpp uses a Semaphore, whereas once_mutex.hpp uses a mutex, the same as the existing boost::thread implementation.
IIRC boost::thread implementation optimizes fast path (get rid of kernel calls) and only briefly holds the named lock. Your stuff looks different in this respect. Is this intentional? regards, alexander.
Alexander Terekhov <terekhov@web.de> writes:
Anthony Williams wrote: [...]
once.hpp uses a Semaphore, whereas once_mutex.hpp uses a mutex, the same as the existing boost::thread implementation.
IIRC boost::thread implementation optimizes fast path (get rid of kernel calls) and only briefly holds the named lock.
Your stuff looks different in this respect. Is this intentional?
No. Thanks for spotting it; optimizing the mutex implementation makes the overall timings ~100 times faster in my tests. I have attached a simple optimised implementation. I am not sure if I can optimize the Semaphore method, since it relies on the Semaphore itself storing the initialized/not flag. I guess that rules it out, then. Anthony -- Anthony Williams Software Developer
Right-oh. Here's two implementations of call_once (attached), for starters.
once.hpp uses a Semaphore, whereas once_mutex.hpp uses a mutex, the same as the existing boost::thread implementation.
Running the test program, the semaphore version gives a timing of 16s compiled with gcc-mingw-4.0.1, and 22s compiled with MSVC 7.1, on my machine. The mutex version gives timings of 23s (gcc) and 29s (MSVC), so the semaphore version is clearly faster.
OK how about a third version: Pros: * Uses only simple atomic operations, easy to implement as a header only solution using Boost's existing shared_ptr support code. * Much faster than either of the alternatives above (see below for timings). * No need to do anything different on CE, or use stringstream etc. * Exception safe etc. * Accepts template functor. Cons: * If the functor takes a long time to execute, and there are multiple threads racing to call-once, then the wait will be less efficient than a mutex, this should be a very rare occurance though. Here's my timings: // Semaphore method: // Elapsed time for one thread=4.816 // Elapsed time for multiple threads=0.05 // // Mutex method: // Elapsed time for one thread=5.387 // Elapsed time for multiple threads=0.05 // // Atomic method: // Elapsed time for one thread=0.01 // Elapsed time for multiple threads=0.06 So the atomic method is only about 500 times faster, in your rather pathological test case :-) I always did wonder why call_once wasn't implemented this way, but never got around to asking, could be I've missed something really obvious of course... ? Thoughts? John.
John Maddock wrote: [...]
* No need to do anything different on CE, or use stringstream etc. ^^
I think that especially on CE, you'd really want to use a lock (on slow path) and never busy-wait... priority inversions, y'know. regards, alexander.
I think that especially on CE, you'd really want to use a lock (on slow path) and never busy-wait... priority inversions, y'know.
Very good point, I knew I'd missed something :-) It would be easy enough to bump up the priority of the thread performing the once-call (the priority ceiling approach), priority inheritance looks more elegant, but is rather harder to implement. John.
John Maddock wrote:
I think that especially on CE, you'd really want to use a lock (on slow path) and never busy-wait... priority inversions, y'know.
Very good point, I knew I'd missed something :-)
You need Sleep(1) instead.
Peter Dimov wrote:
John Maddock wrote:
I think that especially on CE, you'd really want to use a lock (on slow path) and never busy-wait... priority inversions, y'know.
Very good point, I knew I'd missed something :-)
You need Sleep(1) instead.
Won't help. regards, alexander.
Alexander Terekhov wrote:
Peter Dimov wrote:
John Maddock wrote:
I think that especially on CE, you'd really want to use a lock (on slow path) and never busy-wait... priority inversions, y'know.
Very good point, I knew I'd missed something :-)
You need Sleep(1) instead.
Won't help.
Why not?
Peter Dimov wrote:
Alexander Terekhov wrote:
Peter Dimov wrote:
John Maddock wrote:
I think that especially on CE, you'd really want to use a lock (on slow path) and never busy-wait... priority inversions, y'know.
Very good point, I knew I'd missed something :-)
You need Sleep(1) instead.
Won't help.
Why not?
Low priority thread preempted (inside init) by middle priority long running thread which never calls once(). Now comes the high priority thread, calls once()... and sleeps, sleeps, sleeps, ... not good. regards, alexander.
I think that especially on CE, you'd really want to use a lock (on slow path) and never busy-wait... priority inversions, y'know.
Very good point, I knew I'd missed something :-)
You need Sleep(1) instead.
I thought I had, but Oh shucks, apparently not, and I *knew* that as well, how embarrassing! In any case as Alexander pointed out it doesn't completely help due to the priority inversion problem, there's a good description of the issue here: http://www.netrino.com/Publications/Glossary/PriorityInversion.html. It can be worked around, but by the time you've done that, you may well have just have used a mutex and let that handle the thread priorities for you (the previously posted mutex version can be improved upon however, as can the existing Boost.Thread code). John.
On 03/09/05, Alexander Terekhov <terekhov@web.de> wrote:
John Maddock wrote: [...]
* No need to do anything different on CE, or use stringstream etc. ^^
I think that especially on CE, you'd really want to use a lock (on slow path) and never busy-wait... priority inversions, y'know.
Doesn't that just move the race to the lock? One trick might to create a locked mutex in the once state initialization and unlock when initialised. Waiting threads can block on the locked mutex. Or use a condition var...?? matt.
Matt Hurd wrote:
On 03/09/05, Alexander Terekhov <terekhov@web.de> wrote:
John Maddock wrote: [...]
* No need to do anything different on CE, or use stringstream etc. ^^
I think that especially on CE, you'd really want to use a lock (on slow path) and never busy-wait... priority inversions, y'know.
Doesn't that just move the race to the lock?
One trick might to create a locked mutex in the once state initialization and unlock when initialised. Waiting threads can block on the locked mutex.
This just moves the race to the mutex creation.
John Maddock wrote: <snip>
Cons: * If the functor takes a long time to execute, and there are multiple threads racing to call-once, then the wait will be less efficient than a mutex, this should be a very rare occurance though.
Neat. Though , if f was longer than short, and there was contention, a lot of wasted cycles could occur. This leads to the thought that this spinning approach would suffer from a priority problem if sufficient waiting threads, one on a single cpu context, were at a higher priority than the initialising item. This would cause a virtual deadlock, a term I just made up ;-) pthread32's initial implementation was similar. Your atomic approach is the right approach I think. A suitable wait list approach in the "already initializing" section with subsequent processing after initialisation is complete should cure it and keep the essence of the fast approach but make it a fair bit more complicated. matt.
Platform maintainers:
Roland Schwarz: win32 x86 Anthony Williams: win32 x86 Matt Hurd: linux x86_64
matt.
This is possibly a dumb question but is the boost thread library a possibility for use in embedded systems - like WinCE or down to the lower end where I mainly work at. Is the platform somehow abstracted away from most of the code or is it closely tied in and lots of work is needed for each platform and available features are platform dependent? What is the need for boost threads when there's Dinkumware threads ?? - because it's open source and free? Graeme
From: Roland Schwarz <roland.schwarz@chello.at> To: boost@lists.boost.org Subject: Re: [boost] [boost thread rework] New version uploaded to Boost Vault Date: Thu, 01 Sep 2005 09:06:27 +0200
Matt Hurd schrieb:
I'd really like to know that the existing library is being actively and conscientiously maintained before we move on to bigger and better things.
Perhaps not too much to ask for...
I tend to agree with David's sentiment. I'd be happy to start a simplification and restructure of the basic primitives. The code should be restructured into platform specific implementations, like boost has elsewhere (such as the atomic inc for shared_prt), as the current code I find a little tricky to read.
With such a restructure the various platforms could have different maintainers to eliminate a bit of the difficulty of trying to support such a cross platform library.
This is the second time I see this happen :-(
Bill Kempf, the original author of the library silently disappeared. The reason always was unclear to me.
And to all of us, since we haven't been able to get in touch with him.
It looks to me as if he was disappointed that the C++ comitee did not accept anything of the threading work so far. Please correct me if this are wrong impressions.
As far as I know, nothing was ever proposed to the C++ committee, so I don't think this isn't the reason. I could be wrong, though.
At least I learned this from digging through the muddy waters of the mailing list archive.
Next Michael Glassford appeared, trying hard to improve on the library. I don't know what happened. I don't know whether he underestimated the task or he is beeing teared apart by something different. At a minimum I would like to hear what he has to say about the new plans. (I also BCC this to him directly.)
I've been chronically short of time all along. I also recently changed jobs and my interests are moving in other directions. I also don't currently have a machine configured to work on Boost. I'm glad to see interest in improving Boost.Threads, and will be glad to give what help I can, which is obviously pretty limited. I did have some ideas for Boost.Threads that I was working on before changing jobs; perhaps I can refresh myself on them at some point and present the ideas for what they're worth. Mike
"Michael Glassford" <glassfordm@hotmail.com> writes:
It looks to me as if he was disappointed that the C++ comitee did not accept anything of the threading work so far. Please correct me if this are wrong impressions.
As far as I know, nothing was ever proposed to the C++ committee, so I don't think this isn't the reason. I could be wrong, though.
You are not wrong. -- Dave Abrahams Boost Consulting www.boost-consulting.com
Roland Schwarz <roland.schwarz@chello.at> writes: Matt Hurd wrote:
I could pull together a basic posix-based implementation which could be used on win32 with the posix32 lib as well, though a native win32 should happen as most people would find the posix32 layer unacceptable.
I would rather like to see the current interface kept as much as possible.
Matt wasn't talking about changing interface, IIUC, only writing implementation. -- Dave Abrahams Boost Consulting www.boost-consulting.com
On 02/09/05, David Abrahams <dave@boost-consulting.com> wrote: Roland Schwarz <roland.schwarz@chello.at> writes:
Matt Hurd wrote:
I could pull together a basic posix-based implementation which could be used on win32 with the posix32 lib as well, though a native win32 should happen as most people would find the posix32 layer unacceptable.
I would rather like to see the current interface kept as much as possible.
Matt wasn't talking about changing interface, IIUC, only writing implementation.
That's right. Hopefully we can take that momentum and quickly add some new stuff but first things first. rw_mutex might be a candidate for a "simplification" though, we'll see. matt.
Matt Hurd schrieb:
I could pull together a basic posix-based implementation which could be used on win32 with the posix32 lib as well, though a native win32 should happen as most people would find the posix32 layer unacceptable.
I assume you are talking about the licensing issue here do you? Or is it about performance? The pthread port to windows is under a GNU license AFAIK. Regards, Roland
Matt Hurd wrote:
On 01/09/05, David Abrahams <dave@boost-consulting.com> wrote: Ion Gaztañaga <igaztanaga@gmail.com> writes:
I don't want to be a buzz-kill and I know it's not glamorous, but I'm really concerned about the health of the existing Boost.Thread primitives. IIUC, none of these cool high-level components you're working on can act as full replacements for the existing stuff.
In particular, for example, the Boost.Thread documentation needs a lot of work. The code, apparently, needs a rewrite in order to get it to use the Boost license, but that's probably too much to hope for right now.
I'd really like to know that the existing library is being actively and conscientiously maintained before we move on to bigger and better things.
Perhaps not too much to ask for...
I tend to agree with David's sentiment. I'd be happy to start a simplification and restructure of the basic primitives. The code should be restructured into platform specific implementations, like boost has elsewhere (such as the atomic inc for shared_prt), as the current code I find a little tricky to read.
With such a restructure the various platforms could have different maintainers to eliminate a bit of the difficulty of trying to support such a cross platform library.
I could pull together a basic posix-based implementation which could be used on win32 with the posix32 lib as well, though a native win32 should happen as most people would find the posix32 layer unacceptable. Perhaps the current implementation can be refactored to do this, but starting from a clean slate so that the work can be licensed under the boost license might be best.
Level 0 - basic atomic ops, fencing, thread, mutex (normal, recursive, rw), condition - to be replaced and updated with the work happening elsewhere by Lea, Boehm etal. This should also replace the atomic ops used by shared_ptr over time. Should be in a style that suits generic programming via a consistent interface which is currently lacking.
A number of these dealing with sharing objects, such as mutexes, semaphores, signaling events, are useful not merely in the context of threading but also in other contexts, such as interprocess communication, fibers, memory mapping and/or sharing etc. with other uses on different OSs. Might these be better put in their own separate Boost library, and then be used by Boost threading and whatever other Boost libraries might need them without incurring the overhead of linking to the entire threading library ?
"Edward Diener" wrote:
A number of these dealing with sharing objects, such as mutexes, semaphores, signaling events, are useful not merely in the context of threading but also in other contexts, such as interprocess communication, fibers, memory mapping and/or sharing etc. with other uses on different OSs. Might these be better put in their own separate Boost library, and then be used by Boost threading and whatever other Boost libraries might need them without incurring the overhead of linking to the entire threading library ?
Definitely yes. Ion had already implemented these things within Boost.Shmem library (in sandbox). Just someone is needed to pick it up. /Pavel
Pavel Vozenilek wrote:
Ion had already implemented these things within Boost.Shmem library (in sandbox).
Unfortunately it got lost. So Ion needs to upload it again. Or I can if he's not around since I happen to have the 2005-07-09 copy of it.
Just someone is needed to pick it up.
Is it no longer getting supported by Ion? -- -- Grafik - Don't Assume Anything -- Redshift Software, Inc. - http://redshift-software.com -- rrivera/acm.org - grafik/redshift-software.com -- 102708583/icq - grafikrobot/aim - Grafik/jabber.org
"Rene Rivera" wrote:
Pavel Vozenilek wrote:
Ion had already implemented these things within Boost.Shmem library (in sandbox).
Unfortunately it got lost. So Ion needs to upload it again. Or I can if he's not around since I happen to have the 2005-07-09 copy of it.
Just someone is needed to pick it up.
Is it no longer getting supported by Ion?
It is supported but someone may take it out of Shmem and create standalone IPC primitives library. Shmem would get smaller and standalone library more useable. /Pavel
Hi!
Just someone is needed to pick it up.
Is it no longer getting supported by Ion?
I've already uploaded to the new Boost Vault the old version (I've put it in Memory folder). Because of holidays, KDE translations, and thread_ex work, I had no time to improve it, but right now I'm working in a new version, concentrating in simplifying the library, adding sgi slist container, some optimizations, and some new features. I wanted to experiment with SGI ropes, but I've seen this is a lot of work. Of course, the process-shared mutexes, conditions, locks, etc... in this library (I also have a process-shared message-queue) should go in a library with inter-thread synchronization stuff, with the same interface as new boost.threads (or better said, C++ 0x interface). Shmem is big and splitting this part first would make easier to review Shmem. As seen in C++ committee papers, threads are alive, and I hope that there will be a thread/mutex interface proposal soon (I'm big supporter of Kevlin Kenney's interface with improved lockers). Now that I see Shmem is not updated for 3 months, I think it's time to a new version! Regards, Ion
Edward Diener wrote: Matt Hurd wrote:
<snip>
A number of these dealing with sharing objects, such as mutexes, semaphores, signaling events, are useful not merely in the context of threading but also in other contexts, such as interprocess communication, fibers, memory mapping and/or sharing etc. with other uses on different OSs. Might these be better put in their own separate Boost library, and then be used by Boost threading and whatever other Boost libraries might need them without incurring the overhead of linking to the entire threading library ?
It will be interesting to see how much we can implement as header only with the existing interface to prevent the need to link the entire lib. For example, I don't think a basic mutex and scoped lock should need implementation beyond hpp. Being able to deliver suitable primitives for making apps thread safe without linking a boost lib is a nice goal to keep in mind. matt.
"Matt Hurd" wrote:
It will be interesting to see how much we can implement as header only with the existing interface to prevent the need to link the entire lib. For example, I don't think a basic mutex and scoped lock should need implementation beyond hpp. Being able to deliver suitable primitives for making apps thread safe without linking a boost lib is a nice goal to keep in mind.
asio (http://asio.sf/net) has its own threading library (WIn32 + Unix) implemented in headers, for example. /Pavel
I don't want to be a buzz-kill and I know it's not glamorous, but I'm really concerned about the health of the existing Boost.Thread primitives. IIUC, none of these cool high-level components you're working on can act as full replacements for the existing stuff.
I seems that Michael Glassford has no time to actively mantain it, so I think Boost.Threads are just in bug-fixing state. With the implementation thread_ex I just wanted to have a portable implementation of the interface to be presented in the ISO meeting as quickly as possible. Not that I've forgotten the rework, but Kevlin needs also a working (I hope) example. And boost threads is a good basis for that. The proposal is also very incomplete, so we can concentrate now on Boost Threads.
In particular, for example, the Boost.Thread documentation needs a lot of work. The code, apparently, needs a rewrite in order to get it to use the Boost license, but that's probably too much to hope for right now.
I agree. I'm ready to help Matt in reorganizing Boost Threads. Regards, Ion
David Abrahams wrote:
In particular, for example, the Boost.Thread documentation needs a lot of work.
Quite some time ago, I posted a proposal[1] for some clarification regarding the thread-specific pointer. Michael said, he would look at it, once he finds time. I guess this hasn't happened, yet. Regards, m [1] http://lists.boost.org/Archives/boost/2003/11/56503.php Send instant messages to your online friends http://au.messenger.yahoo.com
participants (1)
-
Alexander Terekhov -
Anthony Williams -
David Abrahams -
Edward Diener -
Graeme Prentice -
Ion Gaztañaga -
John Maddock -
Martin Wille -
Matt Hurd -
Michael Glassford -
Pavel Vozenilek -
Peter Dimov -
Rene Rivera -
Rob Stewart -
Roland Schwarz