Is there any value to packages? [was Re: Why no binaries?]
On 10/25/07, Daniel J. Luke <dluke@geeklair.net> wrote:
On Oct 24, 2007, at 10:11 PM, James Sumners wrote:
Why doesn't MacPorts supply binary packages?
No one has worked on it recently.
If you're interested, I'm sure we would be interested in your help.
Is there a status document that addresses where things stand on efforts like this? I haven't been all that successful at building packages within port (port pkg foo where foo is something i would rather not build again on a second machine). I think I may have resorted to taking the output of "port contents" and wrapping it in a tar or zip command, but that doesn't add any of the magic of receipts and the rest of the stuff that makes a ports system worth using. if packages, especially meta-packages, could be licked, it would make things like gimp and gnome a lot more accessible, as there wouldn't be the huge build delay. I'm not a coder of any merit so I can't put my shoulder behind it, but it would be of interest to know if the core team sees any value in packages (possibly as a component of checkpointing the releases: as of a given release, one could be assured that a given subset of essential/popular ports could be installed as source or as packages). It may be one of those things that is nice to have but lack any support on the core team or in the user community. -- Paul Beard / www.paulbeard.org/ <paulbeard@gmail.com/paulbeard@mac.com>
I was mulling this over last night as I was going to sleep. It seems to me that the port maintainer could be relied upon to build the package for his respective port(s). That way, MacPorts doesn't need to hunt for a build machine. Although, someone would need to step up and maintain a cross compiler port for the maintainers. That would make it easier for the port maintainers to build the packages. If they could do something like `port package +g4 +g5 +intel`, I'm sure the idea would go over a lot better. Of course, there would then need to be functionality built into port to pull down binary packages instead of source packages. On 10/26/07, paul beard <paulbeard@gmail.com> wrote:
On 10/25/07, Daniel J. Luke <dluke@geeklair.net> wrote:
On Oct 24, 2007, at 10:11 PM, James Sumners wrote:
Why doesn't MacPorts supply binary packages?
No one has worked on it recently.
If you're interested, I'm sure we would be interested in your help.
Is there a status document that addresses where things stand on efforts like this? I haven't been all that successful at building packages within port (port pkg foo where foo is something i would rather not build again on a second machine). I think I may have resorted to taking the output of "port contents" and wrapping it in a tar or zip command, but that doesn't add any of the magic of receipts and the rest of the stuff that makes a ports system worth using.
if packages, especially meta-packages, could be licked, it would make things like gimp and gnome a lot more accessible, as there wouldn't be the huge build delay. I'm not a coder of any merit so I can't put my shoulder behind it, but it would be of interest to know if the core team sees any value in packages (possibly as a component of checkpointing the releases: as of a given release, one could be assured that a given subset of essential/popular ports could be installed as source or as packages). It may be one of those things that is nice to have but lack any support on the core team or in the user community.
-- Paul Beard / www.paulbeard.org/ <paulbeard@gmail.com/paulbeard@mac.com>
_______________________________________________ macports-users mailing list macports-users@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo/macports-users
-- James Sumners http://james.roomfullofmirrors.com/ "All governments suffer a recurring problem: Power attracts pathological personalities. It is not that power corrupts but that it is magnetic to the corruptible. Such people have a tendency to become drunk on violence, a condition to which they are quickly addicted." Missionaria Protectiva, Text QIV (decto) CH:D 59
On 10/26/07, James Sumners <james.sumners@gmail.com> wrote:
I was mulling this over last night as I was going to sleep. It seems to me that the port maintainer could be relied upon to build the package for his respective port(s). That way, MacPorts doesn't need to hunt for a build machine.
Although, someone would need to step up and maintain a cross compiler port for the maintainers. That would make it easier for the port maintainers to build the packages. If they could do something like `port package +g4 +g5 +intel`, I'm sure the idea would go over a lot better.
Of course, there would then need to be functionality built into port to pull down binary packages instead of source packages.
well, that's a bit ambitious, perhaps. I'd be happy to have a reliable way to make packages and meta packages on my own. I haven't tested it recently, so perhaps it works properly now. If that works, it seems possible to roll out an archive of singleton ports (that aren't part of a cluster or metapackage). The holy grail would be a way to reliably install the MySQL/php/apache cluster or gnome without building anything. Here's an example of trying to make a package from seahorse, which just installed via port upgrade: [/Users/paul]# port pkg seahorse ---> Creating pkg for seahorse-2.20.1 Error: Target org.macports.pkg returned: shell command "mkbom /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_seahorse/work/destroot /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_seahorse/work/seahorse- 2.20.1.pkg/Contents/Archive.bom" returned error 1 Command output: can't stat /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_seahorse/work/destroot (No such file or directory) can't make bom /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_seahorse/work/seahorse- 2.20.1.pkg/Contents/Archive.bom for dir /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_seahorse/work/destroot Error: Status 1 encountered during processing. -- Paul Beard / www.paulbeard.org/ <paulbeard@gmail.com/paulbeard@mac.com>
I missed something in my previous post: the directory for the port had been cleaned, and apparently the pkg command doesn't expect that, nor does it know what to do about it. So it bails. If you build first, than issue the pkg command, it does work, as best I can tell. If you have the super-useful cddir* command in your shell, you can use port build foo && port pkg foo && cddir foo, where foo is the port you just packaged. It will be in the work directory. So perhaps the packaging stuff is closer to reality than I remembered. * port() { case "$1" in cddir) if [ -z "$2" ]; then cd `port echo all | head -n 1 | xargs port dir`/../.. else cd `port dir $2` fi ;; *) /opt/local/bin/port "$@" ;; esac }
I don't understand. The point of binary packages is so that you, the end user, do not have to build them yourself. Thus eliminating the compile errors and making it an easier system to use. On 10/26/07, paul beard <paulbeard@gmail.com> wrote:
On 10/26/07, James Sumners <james.sumners@gmail.com> wrote:
I was mulling this over last night as I was going to sleep. It seems to me that the port maintainer could be relied upon to build the package for his respective port(s). That way, MacPorts doesn't need to hunt for a build machine.
Although, someone would need to step up and maintain a cross compiler port for the maintainers. That would make it easier for the port maintainers to build the packages. If they could do something like `port package +g4 +g5 +intel`, I'm sure the idea would go over a lot better.
Of course, there would then need to be functionality built into port to pull down binary packages instead of source packages.
well, that's a bit ambitious, perhaps. I'd be happy to have a reliable way to make packages and meta packages on my own. I haven't tested it recently, so perhaps it works properly now. If that works, it seems possible to roll out an archive of singleton ports (that aren't part of a cluster or metapackage). The holy grail would be a way to reliably install the MySQL/php/apache cluster or gnome without building anything.
Here's an example of trying to make a package from seahorse, which just installed via port upgrade: [/Users/paul]# port pkg seahorse ---> Creating pkg for seahorse-2.20.1 Error: Target org.macports.pkg returned: shell command "mkbom /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_seahorse/work/destroot /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_seahorse/work/seahorse- 2.20.1.pkg/Contents/Archive.bom" returned error 1 Command output: can't stat /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_seahorse/work/destroot (No such file or directory) can't make bom /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_seahorse/work/seahorse-2.20.1.pkg/Contents/Archive.bom for dir /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_gnome_seahorse/work/destroot
Error: Status 1 encountered during processing.
-- James Sumners http://james.roomfullofmirrors.com/ "All governments suffer a recurring problem: Power attracts pathological personalities. It is not that power corrupts but that it is magnetic to the corruptible. Such people have a tendency to become drunk on violence, a condition to which they are quickly addicted." Missionaria Protectiva, Text QIV (decto) CH:D 59
On Oct 26, 2007, at 13:07, James Sumners wrote:
I was mulling this over last night as I was going to sleep. It seems to me that the port maintainer could be relied upon to build the package for his respective port(s). That way, MacPorts doesn't need to hunt for a build machine.
The user's machine is unlikely to be clean. For example, the user may have other ports installed, and when the user packages a port, that packaged port may inadvertently be depending on some of the user's other ports without his knowledge. Better to have a build machine (or build machines) that can predictably and cleanly create packages.
Although, someone would need to step up and maintain a cross compiler port for the maintainers. That would make it easier for the port maintainers to build the packages. If they could do something like `port package +g4 +g5 +intel`, I'm sure the idea would go over a lot better.
Surely we don't need anything like that. Anyway, it wouldn't be a cross compiler port that we would need. Rather, each port would need to have this capability retrofitted. It would be very similar to the +universal variant we're already trying to retrofit into ports. I would much rather we continue working on perfecting that, rather than introducing ways to cross-compile things, a capability which would probably not be very well tested and therefore buggy.
Of course, there would then need to be functionality built into port to pull down binary packages instead of source packages.
Sure.
On 10/26/07, paul beard wrote:
On 10/25/07, Daniel J. Luke wrote:
On Oct 24, 2007, at 10:11 PM, James Sumners wrote:
Why doesn't MacPorts supply binary packages?
No one has worked on it recently.
If you're interested, I'm sure we would be interested in your help.
Is there a status document that addresses where things stand on efforts like this? I haven't been all that successful at building packages within port (port pkg foo where foo is something i would rather not build again on a second machine). I think I may have resorted to taking the output of "port contents" and wrapping it in a tar or zip command, but that doesn't add any of the magic of receipts and the rest of the stuff that makes a ports system worth using.
if packages, especially meta-packages, could be licked, it would make things like gimp and gnome a lot more accessible, as there wouldn't be the huge build delay. I'm not a coder of any merit so I can't put my shoulder behind it, but it would be of interest to know if the core team sees any value in packages (possibly as a component of checkpointing the releases: as of a given release, one could be assured that a given subset of essential/popular ports could be installed as source or as packages). It may be one of those things that is nice to have but lack any support on the core team or in the user community.
Well, how do you propose building things for different architectures on a build machine without a cross compiler? Unless you have a build machine for each architecture type, PPC (G4 and G5) and Intel, then you would have to use one. I don't much see the point in universal binaries in regard to MacPorts. If you have to build every package yourself, why bother? I certainly don't. It seems to me like a better effort would be to provide pre-built packages ala Debian, et al. On 10/26/07, Ryan Schmidt <ryandesign@macports.org> wrote:
Although, someone would need to step up and maintain a cross compiler port for the maintainers. That would make it easier for the port maintainers to build the packages. If they could do something like `port package +g4 +g5 +intel`, I'm sure the idea would go over a lot better.
Surely we don't need anything like that. Anyway, it wouldn't be a cross compiler port that we would need. Rather, each port would need to have this capability retrofitted. It would be very similar to the +universal variant we're already trying to retrofit into ports. I would much rather we continue working on perfecting that, rather than introducing ways to cross-compile things, a capability which would probably not be very well tested and therefore buggy.
-- James Sumners http://james.roomfullofmirrors.com/ "All governments suffer a recurring problem: Power attracts pathological personalities. It is not that power corrupts but that it is magnetic to the corruptible. Such people have a tendency to become drunk on violence, a condition to which they are quickly addicted." Missionaria Protectiva, Text QIV (decto) CH:D 59
On Oct 26, 2007, at 1:56 PM, paul beard wrote:
Is there a status document that addresses where things stand on efforts like this? I haven't been all that successful at building packages within port (port pkg foo where foo is something i would rather not build again on a second machine).
You can handle that pretty easily with archivemode. I even had a patch kind of working that would look for archivemode style archives on a remote machine (served up via the webserver) before trying to build (and if the archive was there, it would just download it and use it). I haven't really had time to work on the patch to make it acceptable for merging to base/ though.
if packages, especially meta-packages, could be licked, it would make things like gimp and gnome a lot more accessible, as there wouldn't be the huge build delay.
I think we can all agree that that is a good goal. We've had a bunch of different attempts to approach it (archivemode, mp-light, our pre- build packages we offered for download for a while ...), nothing has 'stuck' though. -- Daniel J. Luke +========================================================+ | *---------------- dluke@geeklair.net ----------------* | | *-------------- http://www.geeklair.net -------------* | +========================================================+ | Opinions expressed are mine and do not necessarily | | reflect the opinions of my employer. | +========================================================+
paul beard wrote:
Is there a status document that addresses where things stand on efforts like this? I haven't been all that successful at building packages within port (port pkg foo where foo is something i would rather not build again on a second machine). I think I may have resorted to taking the output of "port contents" and wrapping it in a tar or zip command, but that doesn't add any of the magic of receipts and the rest of the stuff that makes a ports system worth using.
In the recent documentation*, it has been separated between "archives" (tgz,tbz,xar,etc) which are stored versions of the destroot that are used *within* the MacPorts system and "packages" (pkg,rpm,deb,etc) which are packaged for use *outside* the MacPorts system. Both of these two are called "binaries". Both include the same software, though. --anders * http://geeklair.net/new_macports_guide/#using.binaries
paul beard wrote:
I missed something in my previous post: the directory for the port had been cleaned, and apparently the pkg command doesn't expect that, nor does it know what to do about it. So it bails.
If you build first, than issue the pkg command, it does work, as best I can tell.
This is bug #10881: "mpkg/pkg command skips destroot" http://trac.macports.org/projects/macports/ticket/10881 --anders
On Oct 26, 2007, at 16:40, James Sumners wrote:
On 10/26/07, Ryan Schmidt wrote:
Although, someone would need to step up and maintain a cross compiler port for the maintainers. That would make it easier for the port maintainers to build the packages. If they could do something like `port package +g4 +g5 +intel`, I'm sure the idea would go over a lot better.
Surely we don't need anything like that. Anyway, it wouldn't be a cross compiler port that we would need. Rather, each port would need to have this capability retrofitted. It would be very similar to the +universal variant we're already trying to retrofit into ports. I would much rather we continue working on perfecting that, rather than introducing ways to cross-compile things, a capability which would probably not be very well tested and therefore buggy.
Well, how do you propose building things for different architectures on a build machine without a cross compiler? Unless you have a build machine for each architecture type, PPC (G4 and G5) and Intel, then you would have to use one.
I don't much see the point in universal binaries in regard to MacPorts. If you have to build every package yourself, why bother? I certainly don't. It seems to me like a better effort would be to provide pre-built packages ala Debian, et al.
I think we're talking across purposes here. I have no objection to MacPorts eventually providing binaries. I think we're probably a long way from that right now, though. The gcc MacPorts usually uses for most ports -- Apple's gcc -- is a cross-compiler, in that it can create a binary for PowerPC, or for Intel, or for both (universal binary), regardless of what architecture you're on. Traditionally, MacPorts has built native binaries only -- Intel binaries on Intel Macs, PowerPC binaries on PowerPC Macs. Recently, in the MacPorts 1.4.x timeframe, a global +universal variant was added to MacPorts. The purpose is to allow you to build universal binaries of ports, on any Mac. The global +universal variant follows Apple's recommendations, and thus enables this for most standard software. However, many many portfiles are for software that isn't quite standard, so the universal support needs to be tweaked for those portfiles, in either simple or complicated ways. This effort has already been underway for some time, and my point was that we should just continue that effort, so that we eventually have a ports tree that can be built universally. Then, when we gain the ability to distribute binaries some time in the distant future, all we have to do is distribute a single universal binary, which can have been built on any Mac, and will then work on any Mac. This is much better than now introducing lots of new syntax and code for allowing, say, a PowerPC Mac user to build a port for an Intel Mac. An issue we have not yet tackled is the difference between 32-bit and 64-bit machines. Our universal binaries are currently 32-bit only. We could investigate ways to build 4-way universal binaries (32- and 64- bit Intel, 32- and 64-bit PowerPC). I wonder if most software would benefit from being 64-bit, though. I would suspect not, but am not sure. Also, they're for Mac OS X 10.4 and up only. We could also investigate ways to build universal binaries that work on 10.3 and up on PowerPC Macs and 10.4 and up on Intel Macs. But by the time we get around to having binary package distributions, 10.3 may no longer be supported by MacPorts, so it may be a moot point.
Let me make a couple of general comments to help clear up this confusion: On Oct 26, 2007, at 1:56 PM, paul beard wrote:
On 10/25/07, Daniel J. Luke <dluke@geeklair.net> wrote: On Oct 24, 2007, at 10:11 PM, James Sumners wrote:
Why doesn't MacPorts supply binary packages?
No one has worked on it recently.
If you're interested, I'm sure we would be interested in your help.
Is there a status document that addresses where things stand on efforts like this? I haven't been all that successful at building packages within port (port pkg foo where foo is something i would rather not build again on a second machine). I think I may have resorted to taking the output of "port contents" and wrapping it in a tar or zip command, but that doesn't add any of the magic of receipts and the rest of the stuff that makes a ports system worth using.
Building (binary) *packages* (and not "archives" as produced by archivemode and as explained by Anders), is one of MacPorts topmost long term goals, but trust me when I say that doing them right is by far much much more difficult than it sounds: 1) putting together a clean build environment and off it creating reliable, traceable & reproducible builds of a particular port, it's "destroot". A lot of work on this front has gone into MacPorts lately (mainly Eugene Epimenov's "trace mode" improvements) but unfortunately we're still lacking some key functionality to call our software infrastructure "complete" (runtime actions logging and server side processing of the build logs, among other things). I've been very busy with our new website lately but do plan to devote some energy into this in the hopefully not too distant future. 2) Once the build product, the destroot, is done and considered reliable, packaging it into, say, an rpm and/or a deb package is a completely different topic, with its own intricacies and integration issues between MacPorts and the packaging format we still need to sort out. All of this may sound a bit daunting and discouraging, but trust me that that's not my intent. Packaging is one of my personal top goals, but I need to explain that we want to do them right and that's not a simple and quick thing. Also, our our developer force is still unfortunately small and we just don't get enough leg power to move as fast as we'd like. If anyone is interested in helping to materialize this goal sooner than later please do step up, but be warned that we're not at the stage of furnishing the house, but actually looking into the design of the construction as we lay out the foundations. Regards,... -jmpp
I forgot the icing on the cake: On Oct 27, 2007, at 6:20 PM, Juan Manuel Palacios wrote:
Building (binary) *packages* (and not "archives" as produced by archivemode and as explained by Anders), is one of MacPorts topmost long term goals, but trust me when I say that doing them right is by far much much more difficult than it sounds:
1) putting together a clean build environment and off it creating reliable, traceable & reproducible builds of a particular port, it's "destroot". A lot of work on this front has gone into MacPorts lately (mainly Eugene Epimenov's "trace mode" improvements) but unfortunately we're still lacking some key functionality to call our software infrastructure "complete" (runtime actions logging and server side processing of the build logs, among other things). I've been very busy with our new website lately but do plan to devote some energy into this in the hopefully not too distant future.
2) Once the build product, the destroot, is done and considered reliable, packaging it into, say, an rpm and/or a deb package is a completely different topic, with its own intricacies and integration issues between MacPorts and the packaging format we still need to sort out.
3) Package distribution and install time dependency resolving (which is different from build time dependency resolving, what MacPorts currently has) is yet a new dimension we also need to tie into to do packages "the right way". Granted that modern package managers have a sizable bit of this work covered already, but that by no means guarantees that we'll be able to drive blindfolded once we enter that terrain. Again, not-trying-to-discourage-anyone-but-actually-setting-the- record-straight-to-explain-why-we're-taking-so-long-to-deliver- packages.... -jmpp
Juan Manuel Palacios wrote:
2) Once the build product, the destroot, is done and considered reliable, packaging it into, say, an rpm and/or a deb package is a completely different topic, with its own intricacies and integration issues between MacPorts and the packaging format we still need to sort out.
I've fixed the most glaring bugs in the "pkg" and "mpkg" targets, but there are still quite a few remaining... The first remaining issue is to improve the current Jaguar hacks, so that it uses the Leopard packagemanager. (No, this won't fix all the serious shortcomings of Installer.app packages but it will just improve the MacPorts building of them slightly) Most of the others are documented in Trac. --anders
participants (6)
-
Anders F Björklund
-
Daniel J. Luke
-
James Sumners
-
Juan Manuel Palacios
-
paul beard
-
Ryan Schmidt