Boost-wide report -- explicit failure markup not working?
Looking at the boost-wide reports today, testmicrosec_time_clock is marked as failing red on a whole series of compilers. However, they should be 'green' (and have been in past reports) as they have been explicitly marked as failing: http://www.meta-comm.com/engineering/boost-regression/developer/date_time.ht... <test name="testmicrosec_time_clock"> <mark-failure> <toolset name="msvc"/> <toolset name="vc7"/> <toolset name="vc7.1"/> <toolset name="cwpro8"/> <toolset name="borland-5.6.4"/> <toolset name="gcc"/> <toolset name="borland"/> <toolset name="mingw"/> <toolset name="meta-como-win32-4.3.3-msvc"/> <note author="B. Garst" refid="22"/> </mark-failure> </test> Seems like there is something amiss in the results reporting... Jeff
Jeff Garland wrote:
Looking at the boost-wide reports today, testmicrosec_time_clock is marked as failing red on a whole series of compilers. However, they should be 'green' (and have been in past reports) as they have been explicitly marked as failing:
I noted the same thing. http://boost.sourceforge.net/regression-logs/cs-win32_metacomm/developer/pro... has only a couple of yellow cells and boost-wide reports have a huge number of yellow ones. While we're at it, I have two questions about boost-wide regression reports 1. How they are related with individual reports? Are they generated from individual reports. 2. Old style reports showed warnings, for example see: http://boost.sourceforge.net/regression-logs/cs-Linux.html New style reports do not. I find it not very good. I'd really want boost to be warning-free on gcc 3.4 and fixed some errors already. But I'm not sure the above page is up-to-date, and it's hard to search for warning in it. Maybe, new style reports could show warnings too? - Volodya
Vladimir Prus writes:
Jeff Garland wrote:
Looking at the boost-wide reports today, testmicrosec_time_clock is marked as failing red on a whole series of compilers. However, they should be 'green' (and have been in past reports) as they have been explicitly marked as failing:
I noted the same thing.
http://boost.sourceforge.net/regression-logs/cs-win32_metacomm/developer/pro...
has only a couple of yellow cells and boost-wide reports have a huge number of yellow ones.
There *was* a configuration problem with Boost-wide reports that did lead to a stale markup, but in general the comparison above (or at least the wording) is not valid -- there are more platforms in the Boost-wide reports than in any platform-specific ones, and therefore counting overall number of cells of a particular color doesn't make much sense.
While we're at it, I have two questions about boost-wide regression reports
1. How they are related with individual reports? Are they generated from individual reports.
They are generated from the same logs, yes.
2. Old style reports showed warnings, for example see:
http://boost.sourceforge.net/regression-logs/cs-Linux.html
New style reports do not. I find it not very good. I'd really want boost to be warning-free on gcc 3.4 and fixed some errors already. But I'm not sure the above page is up-to-date, and it's hard to search for warning in it. Maybe, new style reports could show warnings too?
Put on the TODO list. -- Aleksey Gurtovoy MetaCommunications Engineering
Jeff Garland writes:
Looking at the boost-wide reports today, testmicrosec_time_clock is marked as failing red on a whole series of compilers. However, they should be 'green' (and have been in past reports) as they have been explicitly marked as failing:
http://www.meta-comm.com/engineering/boost-regression/developer/date_time.ht...
<test name="testmicrosec_time_clock"> <mark-failure> <toolset name="msvc"/> <toolset name="vc7"/> <toolset name="vc7.1"/> <toolset name="cwpro8"/> <toolset name="borland-5.6.4"/> <toolset name="gcc"/> <toolset name="borland"/> <toolset name="mingw"/> <toolset name="meta-como-win32-4.3.3-msvc"/> <note author="B. Garst" refid="22"/> </mark-failure> </test>
Seems like there is something amiss in the results reporting...
Fixed now, thanks for pointing this out! -- Aleksey Gurtovoy MetaCommunications Engineering
Le sam 31/07/2004 à 14:05, Aleksey Gurtovoy a écrit :
Jeff Garland writes:
Looking at the boost-wide reports today, testmicrosec_time_clock is marked as failing red on a whole series of compilers. However, they should be 'green' (and have been in past reports) as they have been explicitly marked as failing:
http://www.meta-comm.com/engineering/boost-regression/developer/date_time.ht...
Seems like there is something amiss in the results reporting...
Fixed now, thanks for pointing this out!
If I understand correctly, failures explicitly marked will be reported in green cells. I'm not sure I like this solution. I would rather the color to be gray. The gray color means "The library author marked it as unusable on particular platform/toolset". It is currently used only for mark-unusable tests; but I think this description equally applies to a mark-failure test, doesn't it? Or another color could be chosen. But having a test suddenly green when a library author becomes aware of the issue is a bit excessive imho. Regards, Guillaume
If we're going to get that, mightn't we also get links for the things that unexpectedly pass (dark green). With so many people coding by experiment, it's sometimes far to easy to accidently put code in a program that compiles on "my compiler" but not on others. Seeing the tests that _should_ fail would help, IMO At Saturday 2004-07-31 06:09, you wrote:
Le sam 31/07/2004 à 14:05, Aleksey Gurtovoy a écrit :
Jeff Garland writes:
Looking at the boost-wide reports today, testmicrosec_time_clock is marked as failing red on a whole series of compilers. However, they should be 'green' (and have been in past reports) as they have been explicitly marked as failing:
http://www.meta-comm.com/engineering/boost-regression/developer/date_time.ht...
Seems like there is something amiss in the results reporting...
Fixed now, thanks for pointing this out!
If I understand correctly, failures explicitly marked will be reported in green cells. I'm not sure I like this solution. I would rather the color to be gray. The gray color means "The library author marked it as unusable on particular platform/toolset". It is currently used only for mark-unusable tests; but I think this description equally applies to a mark-failure test, doesn't it? Or another color could be chosen. But having a test suddenly green when a library author becomes aware of the issue is a bit excessive imho.
Regards,
Guillaume
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Victor A. Wagner Jr. http://rudbek.com The five most dangerous words in the English language: "There oughta be a law"
participants (6)
-
Aleksey Gurtovoy -
Guillaume Melquiond -
Jeff Garland -
Victor A. Wagner Jr. -
Vladimir Prus