On 04.04.2007, at 12:14, Randall Wood wrote:

The idea behind the stable branch is this: When a portfile fails to parse in the unstable branch and causes port installation/upgrades to simply fail (like happened during the recent zlib problem) or causes other ports to break (like happened in the last gettext and libgtkhtml3 upgrades), it would be noticed by users of the unstable branch, and repair activities would prevent it from being picked up by the stable branch, so that once working changes were introduced and the port is left alone more than 2 weeks, it then becomes an upgrade in the stable branch.

Great idea! Two problems: (1) "Repair activities" after something is updated might not involve the offending port itself, but the port that was broken by it, or new compatibility ports made necessary by the update. These repair activities can take a while. Your suggestion, if implemented verbatim, would mean that the offending port (untouched since the problematic update) would become "stable" after two weeks, while the broken ports are still being updated. IOW, the broken ports would become broken in the stable tree as well, and people tracking stable would only be able to fix their systems after "repair activities" have subsided for two weeks. If we want to implement this, we need a "veto" system that would let us flag "offending ports" that are not broken in and of themselves, but still cause problems and breakage in other ports. gettext or guile or libgtkhtml3 would have been flagged "offending", with the flag being removed only after all affected ports have been repaired. Only two weeks after removal of the flag, the port can move to stable. (2) Some changes that affect several ports at once should really be handled as atomic operations. If a new version of aqbanking will only work with a new version of gnucash, updating the one without updating the other may break both. This is just what currently happens, but the existence of a stable branch (implying that things will work if you just track that one) should have a solution for it. Setting and clearing the "veto flag" or touching both ports at the same time might be a feasible workaround. Having versioned dependencies would be even better. Regards, Marc

On Apr 5, 2007, at 3:00 AM, Jordan K. Hubbard wrote:

On Apr 4, 2007, at 3:14 AM, Randall Wood wrote:

...

The idea is this:

Stable and Unstable Ports trees available via both rsync and http

Port currently works by grabbing info out of the port registry and PortIndex file(s) on the local system and by looking at Portfiles in the source trees whenever it is going to change the state of the system by installing/uninstalling/activating/deactivating/ whatever a port.

I think this doesn't go far enough from where we currently are to justify the work involved. I'm much more enthusiastic about taking Kevin's "remote index" idea to its logical conclusion and also, rather than thinking of things in terms of "stable and unstable branches", go to the more powerful model of port attributes which allows for "stability" to be just one possible search/selection criteria.

Thanks for the recap Jordan. And in case anybody is curious, this is pretty the outline of what I've been working on with my mpwa (MacPorts Web App) prototype: - "port submit" to submit a portpkg to the database which gives it a unique pkgid and http: url. (A portpkg is simply a bundled up port directory, plus any additional meta information we need). - A url that may be used to request a particular (previously submitted) portpkg. - The webapp will also generate html UI to describe and index ports. For each port, there is a set of portpkgs for the port, each of which will include additional meta information such as: - submitter, date, etc. - version, epoch, revision - variants, dependencies - tags - Votes (thumbs up/down) - Build and/or install reports (go/no-go status, logs from failed builds) - Jordan describes attributes, I'm thinking tags, since they seem more comfortable in these web 2.0 days, but I think they're basically the same ;). Some tags might be restricted (perhaps "stable"); most will be accessible by all. Tags should also replace the concept of category. But the idea is that you can search and/or browse for ports by tag, and you might also be able to limit installs to only ports that match certain tags, like "stable", or whatever. - Some sort of port dependency closure solver interface. So I can submit a query that says something like "I've got these ports, with these variants installed, and I'd like to end with these ports, with these variants, and perhaps restricted only to the latest versions of ports with these tags: now tell me what actions need to be taken, in which order, to achieve that state." And the answer would come back as a set of actions to be take: "remove these ports, install these ports/variants (and here are their urls) (and here are their binaries)", or "you can't do that, Dave, because you asked for stable ports only and version 3.6 of this port is only available without that tag". The nice part about doing this over the net is that the database of potential portpkgs to install will likely grow far bigger than you want to keep locally. The downside, of course, is that you need net reachability in order to perform the action. A middle ground might be that you could perform the solver query and fetch the needed set of portpkgs while online, and do the actual install while offline. The reality of MacPorts today, of course, is that you mostly need to be online to install anything anyway, since src needs to be fetched. Anyway, those are the ideas I'm playing with. The app I've started writing is in Ruby on Rails, and I'm thinking the database will be MySQL to contain port/portpkg information, metadata, and dependency information. I'm hoping that some pieces of this may come from GSoC: - Dependency analysis given a portpkg (what are the dependencies by given each variant). - New dependency expressions: a dependency might be expressed onto a particular port, plus variants, plus other metadata restrictions: version >= 2.5; variants requires (+server, -client); variants conflicts (apache); tags has (foo,bar); submitter in (jberry,mww); etc. Syntax tbd. - Dependency solver - Reporting of build status/logs back to the webapp. There's some design information here, though it's pretty fluid still: http://trac.macosforge.org/projects/macports/browser/users/jberry/ mpwa/doc/design.txt. Oh, and I guess it goes without saying that once we build the infrastructure do all that, we can also begin to do autobuilds of things and to add binaries for portpkgs. James.

For those who didn't follow the whole remote index discussion way back when it happened, the general concepts were as follows:

1. dports/ as a hierarchy of directories and files goes away. It is replaced by a flat "database" (at least conceptually - the back- end implementation of this database can be done any number of ways, from the crude to the impressive) of port blobs, where a "blob" is basically everything that currently exists in a port directory (Portfile + files/ dir) plus a set of attributes. The most obvious attributes to start with would be name, version and category, but of course there could be any number (including "stable", "devel" or even "unreviewed"), limited only by pragmatism and imagination.

2. sources.conf lists one or more database sources, by http: URL. Since pretty much everyone's firewall allows http access, that gets around that particular problem, though other types of URLs could of course be supported (including, of course, a local database for disconnected operation).

3. Assuming that a database can be contacted, "port install foo" then becomes: 3a). Dispatch a search query for foo 3b). foo is found (let's assume that) and its blob is sent back. 3c). The blob is unpacked and port walks in there to deal with the Portfile (and files/) just as it does today. For all dependencies, of course, repeat steps 3a-3c recursively as needed. 3d). Clean up and remove the blob when finished.

4. Now, since we have a database, other things become possible, such as "port submit". Someone who wants to create a new port and make it available to everyone could, conceivably, go write that port, cd into its directory and say "port submit foo". This would cause foo to be bundled into a blob and shipped off to the remote database, where it would probably (as a matter of policy) be tagged with the "unreviewed" attribute. Anyone else with read access to this database could grab that port immediately thereafter and try it for themselves, so the database becomes a collaborative tool as well.

It's purely a policy decision, of course, but one could even allow for multiple "foos" to exist at the same time, adding some sort of disambiguating attribute, so that several people could submit foo ports and let the reviewers pick the best for stamping "reviewed". We've had people collide in the tree before (usually submitting the same port in two different categories), this would simply make that less traumatic. Needless to say, another policy for "port submit foo" could also be to emit an error if a foo already existed and force the submitter to disambiguate it if they really had a good reason for submitting another port of foo. However we want to go, it's fairly simple to implement the policy once all the other work is done.

Needless to say, if the database is sufficiently powerful to allow for multiple versions of foo to co-exist (and I see no reason why it wouldn't be), the need to maintain dports under svn also goes pretty much by the wayside. The database IS an SCM tool unto itself at that point.

Also pretty much needless to say, a database means that the index is updated whenever something is submitted or has its attributes changed, so there's no need to manually recreate and check in a new PortIndex.

- Jordan

_______________________________________________ macports-dev mailing list macports-dev@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo/macports-dev