[Filesystem] String corruption in path in V3 and Boost 1.44
I have the following method: boost::filesystem::path Sfw::CFolders::GetModulePath() { char buffer[MAX_PATH+1]; chknzapi (GetModuleFileName (NULL, buffer, MAX_PATH)); return buffer; // Converted to fs::path object } Under Boost 1.43 this seemed to work fine. The char buffer is used to create a temporary path object, which is returned by value from the function. But when I upgraded to Boost 1.44, and recompiled with BOOST_FILESYSTEM_VERSION=3 and BOOST_FILESYSTEM_NO_DEPRECATED defined, I found that, although it still compiled, it no longer worked properly. Specifically, the contents of the path becomes corrupted to garbage. I got the impression that the path object returned, instead of containing a copy of the character string, retains a pointer to the original char buffer[] on the stack - which is naturally trampled on by subsequent activity. Changing the function to this: boost::filesystem::path Sfw::CFolders::GetModulePath() { char buffer[MAX_PATH+1]; chknzapi (GetModuleFileName (NULL, buffer, MAX_PATH)); return string(buffer); } seems to fix the problem. I admit I have not examined the code for the filesystem::path class - I am roundly intimidated by Boost code. Is my problem caused by a bug in the library introduced by 1.44, or am I just using the library incorrectly? Is my fixed version acceptable, or am I just 'lucky' that it seems to work? The environment is Visual C++ 10, targetting 32-bit. As well as the #defines listed above, I have also got BOOST_NO_RVALUE_REFERENCES defined, to address a compilation problem with another Boost library. Regards Will PS: Apologies if this is a dupe. I attempted to post this via an NNTP account, which I believed had the correct rights, but didn't seem to do the business. Applied Industrial Systems is a limited company registered in England. Registered number: 2552260. Registered office: 11 Riverview, The Embankment Business Park, Stockport, Cheshire SK4 3GN.
On 22.09.2010 13:15, Will Watts wrote:
I have the following method:
boost::filesystem::path Sfw::CFolders::GetModulePath() { char buffer[MAX_PATH+1]; chknzapi (GetModuleFileName (NULL, buffer, MAX_PATH)); return buffer; // Converted to fs::path object }
Under Boost 1.43 this seemed to work fine. The char buffer is used to create a temporary path object, which is returned by value from the function.
But when I upgraded to Boost 1.44, and recompiled with BOOST_FILESYSTEM_VERSION=3 and BOOST_FILESYSTEM_NO_DEPRECATED defined, I found that, although it still compiled, it no longer worked properly.
Specifically, the contents of the path becomes corrupted to garbage. I got the impression that the path object returned, instead of containing a copy of the character string, retains a pointer to the original char buffer[] on the stack - which is naturally trampled on by subsequent activity.
Changing the function to this:
boost::filesystem::path Sfw::CFolders::GetModulePath() { char buffer[MAX_PATH+1]; chknzapi (GetModuleFileName (NULL, buffer, MAX_PATH)); return string(buffer); }
seems to fix the problem.
I admit I have not examined the code for the filesystem::path class - I am roundly intimidated by Boost code.
Is my problem caused by a bug in the library introduced by 1.44, or am I just using the library incorrectly? Is my fixed version acceptable, or am I just 'lucky' that it seems to work?
[...] I have a similar problem with remove_all(), that is also fixed with a wstring (actually path::string_type) instead of a local TCHAR [] buffer, and also directory iterators look like they are not working correctly in all cases (see a previous article in this group news://news.gmane.org:119/i5qkfu$7ke$1@dough.gmane.org). Trying to build a minimal test case was unsuccessful for me as the bug did not reproduce. Maybe now that I know a local TCHAR [] array may help, I could try again. Anyone else had problems with filesystem v3 paths or iterators ? Constructed from local char arrays ? Will, thanks for the wstring() work-around :) Thank you, Timothy Madden
In article <i7d30l$afa$1@dough.gmane.org>, Timothy Madden wrote:
I have a similar problem with remove_all(), that is also fixed with a wstring (actually path::string_type) instead of a local TCHAR [] buffer,
Aha - so we both think it is probably a bug, then? If nothing else, it is certainly a trap. Do you happen to know if the Filesystem maintainer monitors this list, or if we should be making our case elsewhere? Will
On 22.09.2010 18:10, Will Watts wrote:
In article<i7d30l$afa$1@dough.gmane.org>, Timothy Madden wrote:
I have a similar problem with remove_all(), that is also fixed with a wstring (actually path::string_type) instead of a local TCHAR [] buffer,
Aha - so we both think it is probably a bug, then? If nothing else, it is certainly a trap.
Do you happen to know if the Filesystem maintainer monitors this list, or if we should be making our case elsewhere?
Yes, I think it is a bug. My code also worked with V2 and boost 1.43, and after upgrading to V3 and 1.44 it crashed. I do not know about the Filesystem maintainer, but we need a minimal example to reproduce the problem to make our case elsewhere, that is on the bug tracker or on another list. Timothy Madden
In article <i7db7c$kid$1@dough.gmane.org>, Timothy Madden wrote:
I do not know about the Filesystem maintainer, but we need a minimal example to reproduce the problem to make our case elsewhere, that is on the bug tracker or on another list.
How about this? void main() { const char * f = "foo"; char b[5]; strcpy(b, f); fs::path p1(b); // Debugger shows p1.m_pathname.size = 4 after this fs::path p2(f); // But p2.m_pathname.size = 3 cout << p1 << endl; // prints "foo " cout << p2 << endl; // prints "foo" cout << (p1 == p2) << endl; // prints 1 - they are supposedly identical } Please can you check that I haven't cocked up the strcpy(), or something idiotic! By all means repeat the exercise with wide characters. This doesn't show a spectacular crash, but I think it shows that something is amiss with the C array handling - something to do with the terminating zero byte? What do you reckon? Will
On 22.09.2010 20:36, Will Watts wrote:
In article<i7db7c$kid$1@dough.gmane.org>, Timothy Madden wrote:
I do not know about the Filesystem maintainer, but we need a minimal example to reproduce the problem to make our case elsewhere, that is on the bug tracker or on another list.
How about this?
void main() { const char * f = "foo"; char b[5]; strcpy(b, f); fs::path p1(b); // Debugger shows p1.m_pathname.size = 4 after this fs::path p2(f); // But p2.m_pathname.size = 3 cout<< p1<< endl; // prints "foo " cout<< p2<< endl; // prints "foo" cout<< (p1 == p2)<< endl; // prints 1 - they are supposedly identical }
Please can you check that I haven't cocked up the strcpy(), or something idiotic! By all means repeat the exercise with wide characters.
This doesn't show a spectacular crash, but I think it shows that something is amiss with the C array handling - something to do with the terminating zero byte? What do you reckon?
I think that somehow the template constructor for paths takes the entire array passed as argument and not just the null-terminated string inside the array, which may be smaller. I also got a crash with the attached code on Visual Studio 2008. I have created a track ticket https://svn.boost.org/trac/boost/ticket/4677, lets hope someone picks it up. Timothy Madden
On Wed, Sep 22, 2010 at 13:42, Timothy Madden <terminatorul@gmail.com> wrote:
I think that somehow the template constructor for paths takes the entire array passed as argument and not just the null-terminated string inside the array, which may be smaller.
That's plausible. A number of places in boost like to do that because it allows for embedded nulls in string literals: path p("1234\0abc"); Considering that every filesystem I know of doesn't accept nulls, though, it really doesn't matter for paths. You can get the expected behavour from your code with fs::path p((char const*)b); I suspect it's just that nobody ever considered people using char arrays that weren't string literals, since the "boost way" would have it inside a vector or boost::array, and do fs::path(array.data()) which wouldn't trigger the problem. HTH, ~ Scott
In article <i7dpnt$rqv$1@dough.gmane.org>, Timothy Madden wrote:
I think that somehow the template constructor for paths takes the entire array passed as argument and not just the null-terminated string inside the array, which may be smaller.
This seems to be bang on, as does Scott's speculation as to how it came about. I made a variation of my original experiment, which demonstrates it: const char * f = "foo"; const char * g = "bar"; char b[10]; strcpy(b, f); strcpy(b + strlen(f) + 1, g); // b contains 'f','o','o',0,'b','a','r', 0, Garbage, Garbage fs::path p1(b); // p1.m_pathname.size = 9 fs::path p2(f); // p1.m_pathname.size = 3 // prints "foo bar?" where ? is 8-bit garbage character cout << p1 << endl; // prints "foo" cout << p2 << endl; cout << (p1 == p2) << endl; // prints 1 The last line proves that in some circumstances the nul byte is ignored, in others it is treated as significant. I am surprised the path class can 'know' the length of a C array. I guess that's the magic of templates for me.
I also got a crash with the attached code on Visual Studio 2008. I have created a track ticket https://svn.boost.org/trac/boost/ticket/4677, lets hope someone picks it up.
Top work! Will
On Thu, Sep 23, 2010 at 02:08, Will Watts <willw@applied.co.uk> wrote:
I am surprised the path class can 'know' the length of a C array. I guess that's the magic of templates for me.
It's actually quite simple: template <typename T, size_t N> size_t array_length(T (&)[N]) { return N; } int a[3]; array_length(a) // returns 3 char *p; array_length(p) // doesn't compile The only trick is the necessity to pass the array by reference, since they can't be passed by value. ~ Scott
participants (1)
-
Scott McMurray -
Timothy Madden -
Will Watts