On Oct 3, 2007, at 05:05, Anders F Björklund wrote:
Rainer Müller wrote:
A good reason is, we could adopt new features from base in unstable first and merge them to stable once a new base gets released. For example, removing of "cd" from all ports. Or introduction of compiler.*.
This was what I was thinking about, yes. Or when upgrading one port, but "failing" to upgrade all of the dependencies in the first commit.
Or just to get a little "quarantine" in order to test new releases and updated ports, before introducing them on production machines ?
It sounds great, but are we ready to manage more than one ports tree?
Probably not, as there isn't enough resources maintaining even one tree. I just think it would be a good idea, even if it moved really really slow.
One could start out with a copy of the "archive", and then merge ports one by one from the "trunk" - either manually or maybe just by timer...
I think it's a bad idea, specifically because we're in such a nonoptimal state already. This topic has been discussed on the list before. You may want to look that up in the mailing list archive. Half of our portfiles (2139 of 4300) are currently unmaintained. Even ports that are maintained are not necessarily working properly. How could we in good conscience even declare that the current port collection is "stable"? How would dividing our efforts between stable and unstable branches help us to improve our ports collection faster than we do now? We don't even know which ports currently work and which don't. We don't have any automated build process that tries to build every port on every supported OS & architecture. I kinda feel that would be more useful at this point. We currently get emails or tickets occasionally asking for updates that have already occurred; the user has just forgotten to sync, or the update was just committed and the portindex has not yet been regenerated. If we introduce a quarantine of some sort whereby updates do not immediately appear to users, the frequency of these emails and tickets will increase, and we will have to deal with them, further reducing the amount of time we spend actually fixing the ports. By what mechanism would you suggest that changes move between these two hypothetical ports trees?
Ryan Schmidt wrote: [snip]
Half of our portfiles (2139 of 4300) are currently unmaintained. Even ports that are maintained are not necessarily working properly. How could we in good conscience even declare that the current port collection is "stable"? How would dividing our efforts between stable and unstable branches help us to improve our ports collection faster than we do now?
[snip] I'd gladly volunteer to take some. I think that it would make sense to assign the ports by priority: Important ones first. However, as I'm relatively new to MacPorts I don't know which ports are considered to be important (or very popular). Any suggestions? How would I go about "adopting" an unmaintained port? Michael
On Oct 4, 2007, at 01:37, Michael Wild wrote:
Ryan Schmidt wrote: [snip]
Half of our portfiles (2139 of 4300) are currently unmaintained.
Even
ports that are maintained are not necessarily working properly. How could we in good conscience even declare that the current port collection is "stable"? How would dividing our efforts between stable and unstable branches help us to improve our ports collection faster than we do now?
[snip]
I'd gladly volunteer to take some. I think that it would make sense to assign the ports by priority: Important ones first. However, as I'm relatively new to MacPorts I don't know which ports are considered to be important (or very popular). Any suggestions? How would I go about "adopting" an unmaintained port?
Simply indicate your desire and a committer such as me can change the maintainer address in the port to yours. If you have an update to a port, even an unmaintained or openmaintainer one, attach a diff to a ticket and notify the list and someone will review and commit it.
Ryan Schmidt wrote:
I just think it would be a good idea, even if it moved really really slow.
One could start out with a copy of the "archive", and then merge ports one by one from the "trunk" - either manually or maybe just by timer...
I think it's a bad idea, specifically because we're in such a nonoptimal state already. This topic has been discussed on the list before. You may want to look that up in the mailing list archive.
Took a quick look, but it was mostly about "are you guys crazy". Will take another look...
Half of our portfiles (2139 of 4300) are currently unmaintained. Even ports that are maintained are not necessarily working properly. How could we in good conscience even declare that the current port collection is "stable"? How would dividing our efforts between stable and unstable branches help us to improve our ports collection faster than we do now?
Actually I didn't use the terms "stable" and "unstable" (I think those were from Fink), I used the terms "release" (since the term "archive" means that it is frozen) and "development" (or the SVN term of "trunk").
We don't even know which ports currently work and which don't. We don't have any automated build process that tries to build every port on every supported OS & architecture. I kinda feel that would be more useful at this point.
It's much easier to do packaging and testing, when the rate of change slows down. The automated build and packaging process is being revised now (using it to build RPMS), and that is very useful to have either way.
We currently get emails or tickets occasionally asking for updates that have already occurred; the user has just forgotten to sync, or the update was just committed and the portindex has not yet been regenerated. If we introduce a quarantine of some sort whereby updates do not immediately appear to users, the frequency of these emails and tickets will increase, and we will have to deal with them, further reducing the amount of time we spend actually fixing the ports.
Impatient or out-of-date users will occur either way, the easiest path to help them is probably have an updated ports list for easy viewing - such as a web page with versions and recent updates.
By what mechanism would you suggest that changes move between these two hypothetical ports trees?
"Release Engineering". Eventually, someone will have to take a look at it. But maybe just not today ? --anders
On 04.10.2007, at 08:54, Anders F Björklund wrote:
Ryan Schmidt wrote:
I just think it would be a good idea, even if it moved really really slow.
One could start out with a copy of the "archive", and then merge ports one by one from the "trunk" - either manually or maybe just by timer...
I think it's a bad idea, specifically because we're in such a nonoptimal state already. This topic has been discussed on the list before. You may want to look that up in the mailing list archive.
Took a quick look, but it was mostly about "are you guys crazy". Will take another look...
Half of our portfiles (2139 of 4300) are currently unmaintained. Even ports that are maintained are not necessarily working properly. How could we in good conscience even declare that the current port collection is "stable"? How would dividing our efforts between stable and unstable branches help us to improve our ports collection faster than we do now?
Actually I didn't use the terms "stable" and "unstable" (I think those were from Fink), I used the terms "release" (since the term "archive" means that it is frozen) and "development" (or the SVN term of "trunk").
We don't even know which ports currently work and which don't. We don't have any automated build process that tries to build every port on every supported OS & architecture. I kinda feel that would be more useful at this point.
It's much easier to do packaging and testing, when the rate of change slows down. The automated build and packaging process is being revised now (using it to build RPMS), and that is very useful to have either way.
This is great. I think we can talk about splitting the tree as soon as this features is finished and we have a build-server that reports about build errors.
We currently get emails or tickets occasionally asking for updates that have already occurred; the user has just forgotten to sync, or the update was just committed and the portindex has not yet been regenerated. If we introduce a quarantine of some sort whereby updates do not immediately appear to users, the frequency of these emails and tickets will increase, and we will have to deal with them, further reducing the amount of time we spend actually fixing the ports.
Impatient or out-of-date users will occur either way, the easiest path to help them is probably have an updated ports list for easy viewing - such as a web page with versions and recent updates.
The out-of-date users will be far less if we provide RPMs. The impatient ones are often also those who find the bugs in the first place.
By what mechanism would you suggest that changes move between these two hypothetical ports trees?
"Release Engineering". Eventually, someone will have to take a look at it. But maybe just not today ?
I'd prefer to combine this job with an automated build. This will make this tasks much less error prone and less time consuming. -Markus --- Markus W. Weissmann http://www.mweissmann.de/
Weissmann Markus wrote:
It's much easier to do packaging and testing, when the rate of change slows down. The automated build and packaging process is being revised now (using it to build RPMS), and that is very useful to have either way.
This is great. I think we can talk about splitting the tree as soon as this features is finished and we have a build-server that reports about build errors.
Having a build server is more of an infrastructure/resource question, but there is a lot of work being done with tracing and logging - I'm more doing the oldfashioned approach with a chroot and scripting... But the Koji* build system is kinda nice looking, otherwise... :-)
Impatient or out-of-date users will occur either way, the easiest path to help them is probably have an updated ports list for easy viewing - such as a web page with versions and recent updates.
The out-of-date users will be far less if we provide RPMs. The impatient ones are often also those who find the bugs in the first place.
It's also possible to provides archives (tgz) for impatient developer users, within the port system itself. But both are being built anyway, it's the same destroot contents in both - no matter the wrapping. Having portable MacPorts also helps, easier to build/test on BSD/GNU. --anders * see http://koji.fedoraproject.org/koji/
Top posting to make some general comments. If you guys may remember, I suggested a somewhat large rearrangement of our svn tree a while ago, but all in all I was met with quite a bit of opposition so I simply dropped it (the initiative at the time, not the desire to do it ;-). Basically, my thoughts were along the line of: -) forego the svn {trunk,branches,tags} standard top level dirs model as I believe it doesn't suit us, the only thing we're ever branching and/or tagging is the 'base' component; -) based on that, rearrange as so (leading / indicate I'm referring to top level dirs): /base trunk/ branches/ tags/ /ports (read explanation below) release/ aqua/ audio/ (etc) test/ aqua/ audio/ (etc) <some other tree and children categories> /www <files>, <dirs>. /docs <files>, <dirs>. Most of that layout is self-explanatory, I believe, but my view of the "new" ports tree(s) does warrant some explanation. It was my idea that in making the ports dir a top level one (and thus decoupling it from the erroneously implied "trunk" category), we could leverage the opportunity to create different "sets of trees", the "release" one being the one we distribute to users by default and where "most" things submitted by contributors would go. I don't like the Fink idea of a stable (release) Vs. unstable tree, the last one holding the equivalent of our *-devel ports, as I believe it has been proven that that approach condemns the "stable" tree to an always-out-of-date jail and as a result pushes everyone to the "unstable" tree (a lot of users come to MacPorts complaining Fink only has outdated packages). I did, however, propose a "test" tree of some sort where we would instead test MacPorts itself, freely using whatever new features/ constructs that may be in our trunk base code but not in a MacPorts release at any moment; currently, any new MacPorts feature has to wait for a new release before being adopted in Portfiles, which hinders our ability to test. "mysql5" and "mysql5-devel" ports would both live in the "release" tree if they work with whatever MacPorts release is current, but (a copy of) any of those would have to move to the "test" tree if the corresponding Portfile starts using syntax/ code only available in base's trunk. Users would be free to use the "test" tree against base's trunk or the recommended "release" tree against the current MacPorts release. But that layout is not limited to only those two trees, as any other purpose-specific tree could be created to fulfill whatever need we may have should they arise. For instance, platform specific trees could be created: /ports tiger/ aqua/ audio/ (etc) leopard/ aqua/ audio/ (etc) Do we really need this....? Certainly not now... but who knows if we ever do...? ;-) In any case, having the room to grow is a big plus, I believe, and we don't have it now. And about automated testing through automated ports build runs: this is one of the primary goals I am aiming for in the mid-term future, but there are in my opinion a couple of key components we are still missing to be able to do it properly. One of those is full a blown trace mode so that we can have all the needed information about everything that goes on outside the destroot sandbox; this depends on Eugene's GSoC work so I guess this is a great oportunity to ask for a status update on that (as I did tonight with Chris on IRC ;-). Eugene, could you please bring us up to speed with the state of your work? [1] Secondly.... and obviously... logging support! Can't do any sort of automated software building if we don't have any mean to catch errors and report them. I've spoken at lengths about this already, and since this mail is a tad too long already (were you expecting any different form me? ;-), I'll just point you to the formal proposal I wrote on it: http://trac.macports.org/projects/macports/wiki/PortLoggingProposal As a treat, while evangelizing that to Chris tonight on IRC, he was kind enough to put together a small mock-up of what an xml MacPorts log would look like: http://xml.sfiera.net/macports-log.xml Once I'm done with the new website I plan to devote to some serious energies to this feature that we're so sorely lacking -- do not feel one bit shy to chime in if you feel like helping in any way! And all that having been said, I should be in bed ;-) Regards,... -jmpp [1] I asked Chris for a status update on his GSoC work spanning a few key points: -) original goals; -) if they were modified along the way, how? -) how successfully did you achieve your original and/or modified goals? -) future plans for your end result product(s), most certainly including how you plan to use it (them) in MacPorts Eugene and Elias, you think you could please put something like that together for us? It would greatly help us plan future MacPorts releases. Thank you! On Oct 4, 2007, at 2:54 AM, Anders F Björklund wrote:
Ryan Schmidt wrote:
I just think it would be a good idea, even if it moved really really slow.
One could start out with a copy of the "archive", and then merge ports one by one from the "trunk" - either manually or maybe just by timer...
I think it's a bad idea, specifically because we're in such a nonoptimal state already. This topic has been discussed on the list before. You may want to look that up in the mailing list archive.
Took a quick look, but it was mostly about "are you guys crazy". Will take another look...
Half of our portfiles (2139 of 4300) are currently unmaintained. Even ports that are maintained are not necessarily working properly. How could we in good conscience even declare that the current port collection is "stable"? How would dividing our efforts between stable and unstable branches help us to improve our ports collection faster than we do now?
Actually I didn't use the terms "stable" and "unstable" (I think those were from Fink), I used the terms "release" (since the term "archive" means that it is frozen) and "development" (or the SVN term of "trunk").
We don't even know which ports currently work and which don't. We don't have any automated build process that tries to build every port on every supported OS & architecture. I kinda feel that would be more useful at this point.
It's much easier to do packaging and testing, when the rate of change slows down. The automated build and packaging process is being revised now (using it to build RPMS), and that is very useful to have either way.
We currently get emails or tickets occasionally asking for updates that have already occurred; the user has just forgotten to sync, or the update was just committed and the portindex has not yet been regenerated. If we introduce a quarantine of some sort whereby updates do not immediately appear to users, the frequency of these emails and tickets will increase, and we will have to deal with them, further reducing the amount of time we spend actually fixing the ports.
Impatient or out-of-date users will occur either way, the easiest path to help them is probably have an updated ports list for easy viewing - such as a web page with versions and recent updates.
By what mechanism would you suggest that changes move between these two hypothetical ports trees?
"Release Engineering". Eventually, someone will have to take a look at it. But maybe just not today ?
--anders
_______________________________________________ macports-dev mailing list macports-dev@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo/macports-dev
Juan Manuel Palacios wrote:
If you guys may remember, I suggested a somewhat large rearrangement of our svn tree a while ago, but all in all I was met with quite a bit of opposition so I simply dropped it (the initiative at the time, not the desire to do it ;-).
Wasn't ready for it then, isn't ready for it now. Maybe bring it up again for MacPorts 2.0... :-) I also found the post http://lists.macosforge.org/pipermail/macports-dev/2007-April/ 001159.html
I did, however, propose a "test" tree of some sort where we would instead test MacPorts itself, freely using whatever new features/constructs that may be in our trunk base code but not in a MacPorts release at any moment; currently, any new MacPorts feature has to wait for a new release before being adopted in Portfiles, which hinders our ability to test. "mysql5" and "mysql5-devel" ports would both live in the "release" tree if they work with whatever MacPorts release is current, but (a copy of) any of those would have to move to the "test" tree if the corresponding Portfile starts using syntax/code only available in base's trunk. Users would be free to use the "test" tree against base's trunk or the recommended "release" tree against the current MacPorts release.
This is somewhat similar, except that "test" and "development" moves in opposite directions... i.e. your ports gets pushed from release to "test" when needed for testing features, whereas the "development" tree gets pushed to "release" once released (possibly even getting some testing inbetween, i.e. development -> testing > release) Normally you'd even have a security branch that leads to backporting things for old releases.
But that layout is not limited to only those two trees, as any other purpose-specific tree could be created to fulfill whatever need we may have should they arise. For instance, platform specific trees could be created:
/ports tiger/ aqua/ audio/ (etc) leopard/ aqua/ audio/ (etc)
Do we really need this....? Certainly not now... but who knows if we ever do...? ;-) In any case, having the room to grow is a big plus, I believe, and we don't have it now.
I think that binaries (archives and packages) should be split by OS/major, but I don't think sources (Portfile and files/) should be unless it really really has to. That is, I prefer the "platform {}" variants. http://lists.macosforge.org/pipermail/macports-dev/2007-July/002033.html It doesn't really have to split locally (at least not until the cross-platform development features appear), but it does have to split on the server side when serving out binary packages or binary archives (i.e. pre-compiled). --anders
On Oct 5, 2007, at 4:03 AM, Anders F Björklund wrote:
Juan Manuel Palacios wrote:
If you guys may remember, I suggested a somewhat large rearrangement of our svn tree a while ago, but all in all I was met with quite a bit of opposition so I simply dropped it (the initiative at the time, not the desire to do it ;-).
Wasn't ready for it then, isn't ready for it now. Maybe bring it up again for MacPorts 2.0... :-)
I also found the post http://lists.macosforge.org/pipermail/ macports-dev/2007-April/001159.html
I did, however, propose a "test" tree of some sort where we would instead test MacPorts itself, freely using whatever new features/ constructs that may be in our trunk base code but not in a MacPorts release at any moment; currently, any new MacPorts feature has to wait for a new release before being adopted in Portfiles, which hinders our ability to test. "mysql5" and "mysql5- devel" ports would both live in the "release" tree if they work with whatever MacPorts release is current, but (a copy of) any of those would have to move to the "test" tree if the corresponding Portfile starts using syntax/code only available in base's trunk. Users would be free to use the "test" tree against base's trunk or the recommended "release" tree against the current MacPorts release.
This is somewhat similar, except that "test" and "development" moves in opposite directions...
i.e. your ports gets pushed from release to "test" when needed for testing features, whereas the "development" tree gets pushed to "release" once released (possibly even getting some testing inbetween, i.e. development -> testing > release)
It's a matter of approaches. I just don't like the Fink Stable Vs. Unstable approach and, although not recently, the inability to use unreleased features in our Portfiles is something that has concerned many of us repeatedly, as it severely hinders our testing abilities. Therefore, a "trunk", "testing" or whatever tree for Portfiles that work with base' trunk at any moment (without the constraint of not breaking use installations) seemed like a good approach to me. I should emphasize again that said approach does *not* aim at testing ported software versions, e.g. the latest rpm5 alpha build, but MacPorts unreleased features instead. But,...
Normally you'd even have a security branch that leads to backporting things for old releases.
My proposed model leaves open room to grow any number of port trees we might ever need, including "security" and even Stable Vs. Unstable of for whatever reason we deem it appropriate one day ;-)
---snip---
I think that binaries (archives and packages) should be split by OS/ major, but I don't think sources (Portfile and files/) should be unless it really really has to. That is, I prefer the "platform {}" variants.
Agree, totally platform blocks for the win! I only used the platform trees as examples of what we could do should we ever need to, based on my proposed reorganization. Exporting any of these trees to users would also be as simple as adapting the base/portmgr/mprsyncup script to server them. But,...
http://lists.macosforge.org/pipermail/macports-dev/2007-July/ 002033.html
It doesn't really have to split locally (at least not until the cross-platform development features appear), but it does have to split on the server side when serving out binary packages or binary archives (i.e. pre-compiled).
Yeah, a regular user should only see the need for using a single tree, the one we would serve up by default ("release"), unless very specific needs drove him/her some secondary tree for whatever reason (e.g., a MacPorts developers wanting to test with real life Portfiles a new feature that hasn't been released ;-) But I'll now stop insisting on that, I just liked the idea of being able to create any number of trees should we ever need to. As for binaries, we will need to have platform specific trees, but I don't see those belonging in SVN (ftp, webdav, plain HTTP, whatever, but not SVN) so this particular part of the discussion may be out of scope [1]. Regards,... -jmpp [1] Some people have suggested before we pull the ports tree out of svn entirely, but I disagree with that. Having history tracking for Portfiles and being able to see how they have evolved is a very big plus in my opinion, but that argument does not apply to packages, I believe, as they are simply the end result of the Portfiles... for which we have history tracking ;-) And as for tracing the builds.... you guessed it: http://trac.macports.org/projects/macports/wiki/ PortLoggingProposal (specially the part on post-processing of the logs into tabulated web pages).
2007/10/5, Juan Manuel Palacios <jmpp@macports.org>:
And about automated testing through automated ports build runs: this is one of the primary goals I am aiming for in the mid-term future, but there are in my opinion a couple of key components we are still missing to be able to do it properly. One of those is full a blown trace mode so that we can have all the needed information about everything that goes on outside the destroot sandbox; this depends on Eugene's GSoC work so I guess this is a great oportunity to ask for a status update on that (as I did tonight with Chris on IRC ;-). Eugene, could you please bring us up to speed with the state of your work?
http://trac.macports.org/projects/macports/wiki/soc2007/epimenov
-jmpp
[1] I asked Chris for a status update on his GSoC work spanning a few key points:
-) original goals; -) if they were modified along the way, how? -) how successfully did you achieve your original and/or modified goals? -) future plans for your end result product(s), most certainly including how you plan to use it (them) in MacPorts
Eugene and Elias, you think you could please put something like that together for us? It would greatly help us plan future MacPorts releases. Thank you!
On Oct 4, 2007, at 2:54 AM, Anders F Björklund wrote:
Ryan Schmidt wrote:
I just think it would be a good idea, even if it moved really really slow.
One could start out with a copy of the "archive", and then merge ports one by one from the "trunk" - either manually or maybe just by timer...
I think it's a bad idea, specifically because we're in such a nonoptimal state already. This topic has been discussed on the list before. You may want to look that up in the mailing list archive.
Took a quick look, but it was mostly about "are you guys crazy". Will take another look...
Half of our portfiles (2139 of 4300) are currently unmaintained. Even ports that are maintained are not necessarily working properly. How could we in good conscience even declare that the current port collection is "stable"? How would dividing our efforts between stable and unstable branches help us to improve our ports collection faster than we do now?
Actually I didn't use the terms "stable" and "unstable" (I think those were from Fink), I used the terms "release" (since the term "archive" means that it is frozen) and "development" (or the SVN term of "trunk").
We don't even know which ports currently work and which don't. We don't have any automated build process that tries to build every port on every supported OS & architecture. I kinda feel that would be more useful at this point.
It's much easier to do packaging and testing, when the rate of change slows down. The automated build and packaging process is being revised now (using it to build RPMS), and that is very useful to have either way.
We currently get emails or tickets occasionally asking for updates that have already occurred; the user has just forgotten to sync, or the update was just committed and the portindex has not yet been regenerated. If we introduce a quarantine of some sort whereby updates do not immediately appear to users, the frequency of these emails and tickets will increase, and we will have to deal with them, further reducing the amount of time we spend actually fixing the ports.
Impatient or out-of-date users will occur either way, the easiest path to help them is probably have an updated ports list for easy viewing - such as a web page with versions and recent updates.
By what mechanism would you suggest that changes move between these two hypothetical ports trees?
"Release Engineering". Eventually, someone will have to take a look at it. But maybe just not today ?
--anders
_______________________________________________ macports-dev mailing list macports-dev@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo/macports-dev
Yeah, a regular user should only see the need for using a single tree, the one we would serve up by default ("release"), unless very specific needs drove him/her some secondary tree for whatever reason (e.g., a MacPorts developers wanting to test with real life Portfiles a new feature that hasn't been released ;-)
There's nothing that stops macports developers from having their own trees where they test new features that they're working on. Having another port tree in the repo doesn't help us get more testing if we're not pushing users to it somehow (and I don't know how that would work since we'd also be pushing them to run on pre-release base/) -- Daniel J. Luke +========================================================+ | *---------------- dluke@geeklair.net ----------------* | | *-------------- http://www.geeklair.net -------------* | +========================================================+ | Opinions expressed are mine and do not necessarily | | reflect the opinions of my employer. | +========================================================+
participants (7)
-
Anders F Björklund
-
Daniel J. Luke
-
Juan Manuel Palacios
-
libc
-
Michael Wild
-
Ryan Schmidt
-
Weissmann Markus