Boost.IOStreams and Custom filters on strings
I've written a really simple aggregate_filter that runs the world famous and highly secret ROT13 encryption. This filter works fine when I use a file_sink and file_source. But I really just want to run this on a bunch of string buffers. Unfortunately it does not work, the destination string is always empty. Does anyone understand why? Here's the code, tested against Boost 1.34.1 on OS X 10.5.1: #include <iostream> #include <memory> #include <string> #include <boost/iostreams/copy.hpp> #include <boost/iostreams/device/back_inserter.hpp> #include <boost/iostreams/filter/aggregate.hpp> #include <boost/iostreams/filtering_stream.hpp> template< typename Ch, typename Alloc = std::allocator<Ch> > class basic_rot13_filter : public boost::iostreams::aggregate_filter<Ch, Alloc> { private: typedef boost::iostreams::aggregate_filter<Ch, Alloc> base_type; public: typedef typename base_type::char_type char_type; typedef typename base_type::category category; typedef std::basic_string<Ch> string_type; public: basic_rot13_filter() { } private: typedef typename base_type::vector_type vector_type; void do_filter(const vector_type& src, vector_type& dest) { for (size_t i = 0; i < src.size(); ++i) { dest.push_back(src[i] + 13); } } }; typedef basic_rot13_filter<char> rot13_filter; typedef basic_rot13_filter<wchar_t> wrot13_filter; int main() { std::string src = "SourceText"; boost::iostreams::filtering_istream in; in.push(boost::make_iterator_range(src)); std::string dst; boost::iostreams::filtering_ostream out; out.push(rot13_filter()); out.push(boost::iostreams::back_inserter(dst)); boost::iostreams::copy(in, out); std::cout << dst << std::endl; return 0; }
On Feb 4, 2008 2:03 PM, Stefan Arentz <stefan.arentz@gmail.com> wrote:
void do_filter(const vector_type& src, vector_type& dest) { for (size_t i = 0; i < src.size(); ++i) { dest.push_back(src[i] + 13); } }
Oh please note that I know this is not really rot13. I just needed a simple test and this works fine input like "abcABC" .. the original problem still stands :-) S.
Stefan Arentz wrote:
I've written a really simple aggregate_filter that runs the world famous and highly secret ROT13 encryption. This filter works fine when I use a file_sink and file_source. But I really just want to run this on a bunch of string buffers. Unfortunately it does not work, the destination string is always empty. Does anyone understand why?
I'm not sure why your case doesn't work, but... If you look in libs\iostreams\example directory, you can look at the container_sink.cpp example along with the container_device.hpp which models an STL-compatible sequence as a sink or source.
Stefan Arentz wrote:
I've written a really simple aggregate_filter that runs the world famous and highly secret ROT13 encryption. This filter works fine when I use a file_sink and file_source. But I really just want to run this on a bunch of string buffers. Unfortunately it does not work, the destination string is always empty. Does anyone understand why?
This is the same problem discussed in your other thread 'Another desperate ...'. It can be solved the same way, using reset() or pop(). I guess this would be a good place to ask: As a newcomer to the iostreams library, would you be surprised if copy(in, out) where in and out are filter chains, automatically popped both chains, assuming this behavior was clearly documented? -- Jonathan Turkanis CodeRage http://www.coderage.com
On Feb 7, 2008 4:54 AM, Jonathan Turkanis <turkanis@coderage.com> wrote:
Stefan Arentz wrote:
I've written a really simple aggregate_filter that runs the world famous and highly secret ROT13 encryption. This filter works fine when I use a file_sink and file_source. But I really just want to run this on a bunch of string buffers. Unfortunately it does not work, the destination string is always empty. Does anyone understand why?
This is the same problem discussed in your other thread 'Another desperate ...'. It can be solved the same way, using reset() or pop().
I guess this would be a good place to ask: As a newcomer to the iostreams library, would you be surprised if
copy(in, out)
where in and out are filter chains, automatically popped both chains, assuming this behavior was clearly documented?
Hi Jonathan, For me 'copy' means that data is moved from source to destination without any additional actions from my part. Whether it pops() or resets() the filter internally is ok, as long as it is part of the copy procedure and documented behaviour as you say. I find both operations, pop or reset, very misleading in this case though. They are the last thing you think of when what you really want to do is 'flush' the final bits of the copy. Another thing that would be really nice would be the ability to do something like this: copy(src, base64_decoder() | aes_filter("Secret") | gzip_decompressor(), dst); Or maybe even a shortcut like: std::string src, dst; filter(src, base64_decoder() | ...more ... | ...filters ... | gzip_decompressor(), dst) Since I think a lot of people are operating directly on strings. S.
Stefan Arentz wrote:
On Feb 7, 2008 4:54 AM, Jonathan Turkanis <turkanis@coderage.com> wrote:
Stefan Arentz wrote:
I've written a really simple aggregate_filter that runs the world famous and highly secret ROT13 encryption. This filter works fine when I use a file_sink and file_source. But I really just want to run this on a bunch of string buffers. Unfortunately it does not work, the destination string is always empty. Does anyone understand why? This is the same problem discussed in your other thread 'Another desperate ...'. It can be solved the same way, using reset() or pop().
I guess this would be a good place to ask: As a newcomer to the iostreams library, would you be surprised if
copy(in, out)
where in and out are filter chains, automatically popped both chains, assuming this behavior was clearly documented?
Hi Jonathan,
For me 'copy' means that data is moved from source to destination without any additional actions from my part. Whether it pops() or resets() the filter internally is ok, as long as it is part of the copy procedure and documented behaviour as you say.
Right, I agree that not copying all the data is wrong; but popping the terminal device is surprising as well. I guess I'm just going to have to announce that close() pops a filter chain.
I find both operations, pop or reset, very misleading in this case though. They are the last thing you think of when what you really want to do is 'flush' the final bits of the copy.
I didn't understand this the first time I read it, so let me make sure I get your point: are you saying that 'pop' and 'reset' don't sound like names of functions you would normally call as part of a copy operation, but that the behavior of the functions would be okay after a copy operation, as long as one of them were renamed 'close', or if 'close' were to call one of them? If so, I think I now agree.
Another thing that would be really nice would be the ability to do something like this:
copy(src, base64_decoder() | aes_filter("Secret") | gzip_decompressor(), dst);
Or maybe even a shortcut like:
std::string src, dst; filter(src, base64_decoder() | ...more ... | ...filters ... | gzip_decompressor(), dst)
When I wrote the library I wanted to make copy(src | filter1 | ... | filtern, dest) and copy(src, filter1 | ... | filtern | dest) (and also, as a side-effect copy( src | filter1 | ... | filterm, filter(m+1) | ... | filtern | dest ) ) work as you might expect. Unfortunately I couldn't get it to work on some of the old compilers we supported at the time. (Even though the library seems pretty simple, there's a lot of magic that goes on behind the scenes to allow standard streams, user-defined classes, iterators, and iterator ranges to be handled with the same simple interface. For new compilers it is not much of a challenge, but the old compilers were pushed almost to the breaking point.) It should be pretty easy to implement this now, but it will have to wait to 1.36. I have opened a ticket http://svn.boost.org/trac/boost/ticket/1633.
Since I think a lot of people are operating directly on strings.
S.
Thanks, -- Jonathan Turkanis CodeRage http://www.coderage.com
Jonathan Turkanis wrote:
Stefan Arentz wrote:
For me 'copy' means that data is moved from source to destination without any additional actions from my part. Whether it pops() or resets() the filter internally is ok, as long as it is part of the copy procedure and documented behaviour as you say.
I have opened a ticket http://svn.boost.org/trac/boost/ticket/1633.
I've just closed this ticket. Filtering streams and streambufs are now closable, with an implementation that calls pop(). I haven't yet updated the documentation. -- Jonathan Turkanis CodeRage http://www.coderage.com
Jonathan Turkanis wrote:
I have opened a ticket http://svn.boost.org/trac/boost/ticket/1633.
The correct ticket is http://svn.boost.org/trac/boost/ticket/1624. Sorry for any confusion. -- Jonathan Turkanis CodeRage http://www.coderage.com
participants (1)
-
eg -
Jonathan Turkanis -
Stefan Arentz