Re:Re: [boost] Re: 1.32 release preparation
Message: 16 Date: Sun, 27 Jun 2004 22:25:47 -0500 From: "Aaron W. LaFramboise" <aaronrabiddog51@aaronwl.com> Subject: Re: [boost] Re: 1.32 release preparation To: boost@lists.boost.org Message-ID: <40DF8FBB.4040408@aaronwl.com> Content-Type: text/plain; charset=us-ascii Aaron W. LaFramboise wrote:
Robert Ramey wrote:
Dave Abrahams wrote:
I'd like to know the reasons for "all the sudden rush". We did announce a schedule long ago, and it has already been delayed by three weeks. I'm not asking in order to point fingers; I just want to know how to avoid the "sudden rush" next time.
Here is the current scenario.
a) The announcement gets posted - release 1.32 scheduled for date x. b) Each developer decides - uh oh I better get in my changes before date x. c) Of course these changes take longer d) and break some other things e) which takes longer still
Personally I think the concept of scheduling a release fundamentally flawed. I would suggest another idea.
As an observer, I'm fairly impressed with GCC's release methodology. http://gcc.gnu.org/develop.html
A critical component is a schedule that includes several stages during which certain sorts of changes are unacceptable.
There's still rush, but most of the rush for major features or other destabilizing elements happens long before the release comes around.
I found this to be very interesting. I had never seen this before. It basically implements my view that the "Main Line" should be code that is always works and is ready to be used, tested and potentially released. As far as I'm concerned it's totally consistent with my proposal which I thought was un-orthodox but now see that is used at least by GCC. So, my view would be - stick to the original 28 June date, require that new libraries be added to branches and merge to main as appropriate. I realize its not THAT easy - but it would address the concerns that I have about the whole process and allow all libraries to proceed on independent schedules. New libraries would be tested against the "known good tree" independently of one another. Take as a small example changes in MPL. The current method is to check in a new version of MPL and see what happens. Presumable some issues will arise and now we have a broken system which inhibits development of other libraries until the issues are resolved. Under the proposed method, the new MPL would be check in under its own branch and issues would be detected on all libraries before they affect other developers. Anyway - my 2 cents. Robert Ramey
"Robert Ramey" <ramey@rrsd.com> writes:
As an observer, I'm fairly impressed with GCC's release methodology. http://gcc.gnu.org/develop.html
A critical component is a schedule that includes several stages during which certain sorts of changes are unacceptable.
There's still rush, but most of the rush for major features or other destabilizing elements happens long before the release comes around.
I found this to be very interesting. I had never seen this before. It basically implements my view that the "Main Line" should be code that is always works and is ready to be used, tested and potentially released.
Did you miss this part? Schedule Development on our main branch will proceed in three stages. Each stage will be two months in length. Stage 1 During this period, changes of any nature may be made to the compiler. I think they are willing to destabilize the main trunk at least during stage 1.
As far as I'm concerned it's totally consistent with my proposal which I thought was un-orthodox but now see that is used at least by GCC.
So, my view would be - stick to the original 28 June date, require that new libraries be added to branches and merge to main as appropriate.
I don't think there's any way we can do a release _today_, even with no new libraries ;-)
I realize its not THAT easy - but it would address the concerns that I have about the whole process and allow all libraries to proceed on independent schedules. New libraries would be tested against the "known good tree" independently of one another.
Take as a small example changes in MPL. The current method is to check in a new version of MPL and see what happens. Presumable some issues will arise and now we have a broken system which inhibits development of other libraries until the issues are resolved.
Under the proposed method, the new MPL would be check in under its own branch and issues would be detected on all libraries before they affect other developers.
First of all, a new version of MPL is the main motivation for this particular release, so we _are_ going to merge it in from its development branch (mplbook) before the trunk branches for release. Secondly, a core library like MPL may occasionally change its interface in ways that are not strictly backwards-compatible. If development of other libraries continues while MPL is on its branch, it may be unreasonable to expect MPL's branch to ever work with all those libraries' trunk state. At some point, you have to check it in and let the other libraries adjust. I like the idea of keeping the trunk healthy -- that's the way I always try to operate -- but I don't think that branching for release without warning or communication is reasonable, and I think you have to occasionally be willing to have major changes to core libraries moved to the trunk before you can guarantee that it will cause no trunk breakage. Boost is a collaborative effort, and we rely on communication and cooperation. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
Schedule
Development on our main branch will proceed in three stages. Each stage will be two months in length.
Stage 1
During this period, changes of any nature may be made to the compiler.
I think they are willing to destabilize the main trunk at least during stage 1.
It's worth pointing out that actual breakage itself is never allowed. All patches must pass regression testing before they can be committed, even in Stage 1. Very rarely is a patch accepted which causes a regression or other failure. Also, it is never ever acceptable for a patch to be committed that breaks bootstrap or some other major functionality, as seems to happen occasionally with Boost, as this prevents actual work or testing from happening. The previous sorts of broken code are what branches are for. The destablization allowed in stage 1 is primarily code which passes the regression tests, but is so large or intrusive that there are almost certainly unknown significant bugs or unforeseen problems. Aaron W. LaFramboise
"Aaron W. LaFramboise" <aaronrabiddog51@aaronwl.com> writes:
I think they are willing to destabilize the main trunk at least during stage 1.
It's worth pointing out that actual breakage itself is never allowed. All patches must pass regression testing before they can be committed, even in Stage 1. Very rarely is a patch accepted which causes a regression or other failure. Also, it is never ever acceptable for a patch to be committed that breaks bootstrap or some other major functionality, as seems to happen occasionally with Boost, as this prevents actual work or testing from happening.
The previous sorts of broken code are what branches are for.
The destablization allowed in stage 1 is primarily code which passes the regression tests, but is so large or intrusive that there are almost certainly unknown significant bugs or unforeseen problems.
So what's in the regression tests? Do they test every single compiler back-end? Do they have one of every target machine? It seems to me that changes in the intermediate language or even some front-ends might be very difficult to make if they have to keep every host/target combination working. IIUC, people who maintain ports to various targets are domain experts much like the maintainers of various "higher-level" boost libraries. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
"Aaron W. LaFramboise" <aaronrabiddog51@aaronwl.com> writes:
I think they are willing to destabilize the main trunk at least during stage 1.
It's worth pointing out that actual breakage itself is never allowed. All patches must pass regression testing before they can be committed, even in Stage 1. Very rarely is a patch accepted which causes a regression or other failure. Also, it is never ever acceptable for a patch to be committed that breaks bootstrap or some other major functionality, as seems to happen occasionally with Boost, as this prevents actual work or testing from happening.
The previous sorts of broken code are what branches are for.
The destablization allowed in stage 1 is primarily code which passes the regression tests, but is so large or intrusive that there are almost certainly unknown significant bugs or unforeseen problems.
So what's in the regression tests? Do they test every single compiler back-end? Do they have one of every target machine? It seems to me that changes in the intermediate language or even some front-ends might be very difficult to make if they have to keep every host/target combination working. IIUC, people who maintain ports to various targets are domain experts much like the maintainers of various "higher-level" boost libraries.
Almost everything is the regression tests. The idea is, that if your changes pass the regression test, everything that worked before should still work. What might have problems would be new features that don't have tests, or odd bugs that noone has written tests for, etc. There is a certain amount of civic responsibility here. The idea is that a submitter is responsible for his patch. If a patch breaks something, even if its only by exposing a latent bug, the submitter has an obligation to help the maintainer of the broken area to fix it. It's my opinion that this works well in practice. I personally beleive its very important that the main tree should always be working, even if it is "unstable" simply due to its sheer volatility. If patches are allowed to be checked in that cause regressions, and the submitter is not obligated to fix the problems created by the introduction of the patch, there is no particular garentee that it will be fixed at all in a timely manner. If parts of the project are broken for extended periods of time, development work might be dramatically slowed or halted altogether. On the other hand, theres a lot of momentum to get new features out there and integrated. Branches work very well for this. In GCC, there are several branches that depend on other branches, all for functionality that is eventually going to make it into mainline (after it works!!). Everyone gets to work on their code without fear that someone else might break it and temporarily halt their work, and everyone is happy. Aaron W. LaFramboise
On Monday 28 June 2004 4:32 pm, Aaron W. LaFramboise wrote:
Almost everything is the regression tests. The idea is, that if your changes pass the regression test, everything that worked before should still work. What might have problems would be new features that don't have tests, or odd bugs that noone has written tests for, etc.
There is a certain amount of civic responsibility here. The idea is that a submitter is responsible for his patch. If a patch breaks something, even if its only by exposing a latent bug, the submitter has an obligation to help the maintainer of the broken area to fix it. It's my opinion that this works well in practice.
GCC has an absolutely _wonderful_ automatic regression tester. If you check in code and something breaks, it will e-mail you (and everyone else that checked in code since the last working run) and tell you _precisely_ what test cases broke. Then it will keep e-mailing you until it gets fixed. Although it can get annoying when it isn't your bug (you just happened to commit when someone else did), it does work quite well. Without a system such as this, it's hard to even _know_ when you've broken something (although the Metacomm regression test results _really_ help here). Doug
"Aaron W. LaFramboise" <aaronrabiddog51@aaronwl.com> writes:
Almost everything is the regression tests. The idea is, that if your changes pass the regression test, everything that worked before should still work. What might have problems would be new features that don't have tests, or odd bugs that noone has written tests for, etc.
There is a certain amount of civic responsibility here. The idea is that a submitter is responsible for his patch. If a patch breaks something, even if its only by exposing a latent bug, the submitter has an obligation to help the maintainer of the broken area to fix it. It's my opinion that this works well in practice.
This isn't a new idea for Boost: that's the way we've always operated. The difference is probably that GCC has a nice automatic regression test system. Several people have been working on setting that up for Boost, but it's not done yet.
I personally beleive its very important that the main tree should always be working, even if it is "unstable" simply due to its sheer volatility. If patches are allowed to be checked in that cause regressions, and the submitter is not obligated to fix the problems created by the introduction of the patch, there is no particular garentee that it will be fixed at all in a timely manner. If parts of the project are broken for extended periods of time, development work might be dramatically slowed or halted altogether.
It's never been OK to leave things in a broken state. IMO the one possible exception is a new library that nothing depends on and is not yet officially released. It might be OK to allow that library to be "broken" while the maintainer works on porting issues. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
The XLS-based regression reports have been re-worked to improve their usability and informativeness. In particular, the detailed report page is now spit into seperate pages, one per library, mainly to reduce the load time. The whole thing can be seen at http://tinyurl.com/3a45y. Comments on a further directions for improvement, as well as reports of bugs and broken links are welcome and appreciated. The broken logo and test source links will be fixed in the next run. -- Aleksey Gurtovoy MetaCommunications Engineering
Aleksey Gurtovoy wrote:
The XLS-based regression reports have been re-worked to improve their usability and informativeness. In particular, the detailed report page is now spit into seperate pages, one per library, mainly to reduce the load time. The whole thing can be seen at http://tinyurl.com/3a45y.
Comments on a further directions for improvement, as well as reports of bugs and broken links are welcome and appreciated. The broken logo and test source links will be fixed in the next run.
Some comments: 1. The "fail" entries in user summary used to be white before, and now are green. I think that the previous behaviour was better, now fail entries are hard to find. 2. The user summary for program options shows gray boxes with the "unexp" text. I think this should be "n/a", according to the legend. 3. The linux regression results looks very dump now -- no colors, not table decorations, and so on. I'm not sure if this is related to your changes -- but looks like missing CSS somewhere. This happens both in Konqueror and Mozilla. - Volodya
On Tue, Jun 29, 2004 at 09:47:57AM +0400, Vladimir Prus wrote:
Some comments:
1. The "fail" entries in user summary used to be white before, and now are green. I think that the previous behaviour was better, now fail entries are hard to find.
i think they are green because they are intended failures so they need not to pop into the eyes regards, gernot kerschbaumer
Gernot Kerschbaumer wrote:
On Tue, Jun 29, 2004 at 09:47:57AM +0400, Vladimir Prus wrote:
Some comments:
1. The "fail" entries in user summary used to be white before, and now are green. I think that the previous behaviour was better, now fail entries are hard to find.
i think they are green because they are intended failures so they need not to pop into the eyes
That's good for *developer* summary. But for user summary, I'd really want the user to see the "fail" entries. E.g. program_options/variables_map/vc6 failure is serious, and I'd want the users to see it, because otherwise they might think the library is fully usable on vc6, which is not the case. - Volodya
Vladimir Prus writes:
Aleksey Gurtovoy wrote:
The XLS-based regression reports have been re-worked to improve their usability and informativeness. In particular, the detailed report page is now spit into seperate pages, one per library, mainly to reduce the load time. The whole thing can be seen at http://tinyurl.com/3a45y.
Comments on a further directions for improvement, as well as reports of bugs and broken links are welcome and appreciated. The broken logo and test source links will be fixed in the next run.
Some comments:
1. The "fail" entries in user summary used to be white before, and now are green. I think that the previous behaviour was better, now fail entries are hard to find.
Hmm, these used to be yellow, actually. They turned white accidently, and I should have checked for the original color before fixing it. Put on the TODO list, thanks. On a side note, if you provide a note that explains each such failure, it will both make them more visible *and* actually give user a clue how serious the problem is.
2. The user summary for program options shows gray boxes with the "unexp" text. I think this should be "n/a", according to the legend.
Yep, thanks.
3. The linux regression results looks very dump now -- no colors, not table decorations, and so on. I'm not sure if this is related to your changes
It is. In my annoucement I totally forgot to mention that both the XSL reports upload procedure and the report links need to be adjusted to the new directory structure. [Sorry, Martin!]
-- but looks like missing CSS somewhere. This happens both in Konqueror and Mozilla.
It's worse than just a missing CSS -- if you look at the report dates, you'll notice that these are left-overs from the old format. The new pages are there, though -- you just have to access them directly: http://boost.sourceforge.net/regression-logs/cs-Linux/developer/index.html http://boost.sourceforge.net/regression-logs/cs-Linux/user/index.html Thank you for reporting these! -- Aleksey Gurtovoy MetaCommunications Engineering
Aleksey Gurtovoy wrote:
Vladimir Prus writes:
Aleksey Gurtovoy wrote:
The XLS-based regression reports have been re-worked to improve their usability and informativeness. In particular, the detailed report page is now spit into seperate pages, one per library, mainly to reduce the load time. The whole thing can be seen at http://tinyurl.com/3a45y.
Comments on a further directions for improvement, as well as reports of bugs and broken links are welcome and appreciated. The broken logo and test source links will be fixed in the next run.
Some comments:
1. The "fail" entries in user summary used to be white before, and now are green. I think that the previous behaviour was better, now fail entries are hard to find.
Hmm, these used to be yellow, actually. They turned white accidently, and I should have checked for the original color before fixing it. Put on the TODO list, thanks.
On a side note, if you provide a note that explains each such failure, it will both make them more visible *and* actually give user a clue how serious the problem is.
I believe there are notes for all failures, except for unusable msvc-stlport. See, for example: http://tinyurl.com/2uyug The notes for individual failures do not show up in the summary page; I though it was intended this way.
3. The linux regression results looks very dump now -- no colors, not table decorations, and so on. I'm not sure if this is related to your changes
It is. In my annoucement I totally forgot to mention that both the XSL reports upload procedure and the report links need to be adjusted to the new directory structure. [Sorry, Martin!]
-- but looks like missing CSS somewhere. This happens both in Konqueror and Mozilla.
It's worse than just a missing CSS -- if you look at the report dates, you'll notice that these are left-overs from the old format. The new pages are there, though -- you just have to access them directly:
http://boost.sourceforge.net/regression-logs/cs-Linux/developer/index.html http://boost.sourceforge.net/regression-logs/cs-Linux/user/index.html
Thanks, I'll look at those.... - Volodya
Aleksey Gurtovoy wrote:
Vladimir Prus writes:
Aleksey Gurtovoy wrote: ... 3. The linux regression results looks very dump now -- no colors, not table decorations, and so on. I'm not sure if this is related to your changes
It is. In my annoucement I totally forgot to mention that both the XSL reports upload procedure and the report links need to be adjusted to the new directory structure. [Sorry, Martin!]
FWIW, I updated the the upload procedure and the links right after I received your first message regarding the chance. Regards, m
Martin Wille writes:
Aleksey Gurtovoy wrote:
Vladimir Prus writes:
Aleksey Gurtovoy wrote: ... 3. The linux regression results looks very dump now -- no colors, not table decorations, and so on. I'm not sure if this is related to your changes It is. In my annoucement I totally forgot to mention that both the XSL reports upload procedure and the report links need to be adjusted to the new directory structure. [Sorry, Martin!]
FWIW, I updated the the upload procedure and the links right after I received your first message regarding the chance.
Oh, I see -- you are linking both the new and the old-style reports. Sorry for the confusion. The detailed old-style pages are broken by the new CSS, though. -- Aleksey Gurtovoy MetaCommunications Engineering
Aleksey Gurtovoy wrote:
Oh, I see -- you are linking both the new and the old-style reports. Sorry for the confusion. The detailed old-style pages are broken by the new CSS, though.
Why are the old-style pages being generated then? Regards, m
Martin Wille writes:
Aleksey Gurtovoy wrote:
Oh, I see -- you are linking both the new and the old-style reports. Sorry for the confusion. The detailed old-style pages are broken by the new CSS, though.
Why are the old-style pages being generated then?
They are not! At least not by the stylesheets/scripts that are in the CVS. Since you are referencing them, I assumed you were generating these using the old scripts. Now that I look at their date more carefully, I see that they are just stale results which can be safely deleted. -- Aleksey Gurtovoy MetaCommunications Engineering
participants (2)
-
Aaron W. LaFramboise -
Aleksey Gurtovoy -
David Abrahams -
Doug Gregor -
Gernot Kerschbaumer -
Martin Wille -
Robert Ramey -
Vladimir Prus