Hello Ryan, good to see some feedback! ;-) On Apr 24, 2007, at 8:02 PM, Ryan Schmidt wrote:
What is the deal with the PortSystem value? :) When does that get incremented? What happens if/when it does?
The PortSystem declaration is (was) meant to be a floating point value stating what "version" of the "port API", the portal to the library that parses and executes the Portfiles, is needed to do exactly that, parse and execute a given Portfile. However, the only value that was ever offered (when DarwinPorts first started) is 1.0 (I think at some point, maybe before going public, a 0.5 value also existed), which is why you see every single Portfile with that value and a base/src/port1.0 directory in svn. Another proposal I've discussed on some opportunities with people like Kevin, James and Landon is putting that functionality to work to effectively start using the PortSystem declaration, designing a strategy to bump its value as needed (in tandem with the library version) and catching it when first parsing a Portfile to determine if the installed version of MacPorts can handle it. The base/doc/INTERNALS file also explains some of what I've written here.
Users and committers would be free to suggest the creation of whatever justifiable new tree they find appropriate. Making all of them available through svn and sync (rsync) and whatever else would be immediately beneficial to all, I'm sure it's easy to appreciate.
I can't think of any other trees we would need at this point, and I would even discourage the creation of arbitrary trees at anyone's discretion, without some compelling reason. But at least the directory structure you propose would certainly enable an arbitrary number of trees, should we find we need them.
Agreed on all of this. We would not be creating arbitrary trees at will, not at all, but actually only after compelling reasons for them are provided and wide consensus is reached. I only said users would be free to "suggest" a given tree ;-)
Such rearrangements to our repo would of course require changes in our source code to upgrade existing installations to leverage them (for instance, the selfupdate procedure hardcodes the path to the sources and therefore would break if we change that under the hood c.f. branches/release_1_4 initially missing the base directory level), but I believe the results will be well worth the effort. What I would like to do here is reach consensus on the best possible repo redesign (which I'm hoping does take place!) so that we see ourselves in the need to upgrade users only once.
Does selfupdate get the new sources directly from the Subversion repository?
Nope, both sync and selfupdate get their feeds from an rsync repo.
Somehow I thought rsync was used for that too, and that there was therefore some level of abstraction between the repo and the selfupdate command such that reorganizations of the repository would not pose a problem.
Yes and no. An explanation is in order, I feel. Rsync repos used by MacPorts still live on OpenDarwin servers, sampson I believe, where the base/portmgr/mprsyncup script is installed and working day in and day out (it's a rudimentary script that could see many improvements, agreed, but I would only delve into improving it if we do go forward with all these reorganizations I'm proposing). Said script checks out all of trunk and puts it in a given path that the rsync server makes available as the dpupdate module; it then does some sh & svn magic and checks out the tag listed in base/config/RELEASE_URL (listing our official release) and makes available at another path that the rsync server publishes as the dpupdate1 module. That's why we ship "dpupdate1/base" as the default value of the 'rsync_dir' key in ports.conf (used by the selfupdate target) and "dpupdate/dport" as the default value in sources.conf (used by both sync and selfupdate). Switching to "dpupdate/base" for 'rsync_dir' will update you to trunk through selfupdate, but that's as much flexibility as we currently have. For one, changing these values server side in any way would surely mandate changing our default conf values to follow the new modules, and of course providing a migration path for existing installations. Secondly, the "seflupdate" target is rather fragile in its coding; from base/src/darwinports1.0/darwinports.tcl, line No. 1501: if { [catch { system "cd $dp_base_path && ./configure --prefix= $prefix --with-install-user=$owner --with-install-group=$group --with- tclpackag e=$tclpackage && make && make install" } result] } Where $dp_base_path is set to (line No. 1438): set dp_base_path [file join $prefix var/db/dports/sources/rsync.$ {rsync_server}_${rsync_dir}/] Therefore, on a default MacPorts installation, $dp_base_path maps to /opt/local (${prefix}) + var/db/dports/sources/ + rsync.rsync.darwinports.org (rsync.${rsync_server}) + _dpupdate1/base (_${rsync_dir}), "/opt/local/var/db/dports/sources/ rsync.rsync.darwinports.org_dpupdate1/base/" in all. The selfupdate procedure cd's into that dir and looks for the configure script to do its magic, meaning that if the exact path changes in any way the whole thing breaks irrecoverably (I here again refer to how the release_1_4 branch had to be recreated for selfupdate to work after I omitted the "base" directory level by mistake when first creating it). With the redesigned repo we would no longer have trunk/base but actually base/{trunk,branches,tags}, so paths could become mangled in various ways we might not even be aware of (we're not even considering how the rsync server on sampson exports the modules, since we don't have access to that conf file). Other than migrating users to new paths and rsync moduels, I would like to have this inherent fragility corrected in an upgraded selfupdate procedure.
If it currently does come directly out of the repo, it might be beneficial to separate that conceptually, so that reorganizations of the repository do not break any future selfupdate procedure.
Addressed above.
Do we currently have the ability to configure our MacPorts installations such that selfupdate will download and build the latest version from trunk, rather than from a released branch? That's a capability that I think would be good to have (or keep) for developers.
Addressed above.
Lastly, the doc and www dirs currently inside trunk could be moved to a "legacy" (detention!) top level dir too (and pulled out of it once brought into the light again ;-)
repository/legacy/docs/{www,faq,guide}
Moving www, faq and guide to a legacy directory sounds like a good idea to me, since it makes it clear that these parts are no longer in active development nor do they represent the current state of, for example, the web site.
A good point I wasn't too clear on when first posting, thanks for expanding on it.
I don't think there should be a "docs" directory, however. I recommend:
repository/legacy/faq repository/legacy/guide repository/legacy/www
There already exists a docs dir (trunk/docs), but I made a mistake when making my proposal for this. I wrote "repository/legacy/docs/ {www,faq,guide}" when it should have been "repository/legacy/ {docs,www,faq,guide}". Thanks for pointing me to my mistake!
PS: Yes, I'm also emphasizing the new ports directory should be called just that, "ports", and not "dports" nor "mports" :-P
Agreed.
Again, thanks for the feedback! Regards,... -jmpp