hosting an internal macports server with binaries
Is it possible to have an internal macports mirror that also contains binaries, so I can compile all the ports I need once and install them on several boxes instead of re-compiling everything on every single box? Thanks, Rick Gigger
Rick Gigger wrote:
Is it possible to have an internal macports mirror that also contains binaries, so I can compile all the ports I need once and install them on several boxes instead of re-compiling everything on every single box? Great question. And even before the question of a local mirror: is there a format for packaging a set of binaries that can then be installed into an existing system -- like a *.deb or *.rpm file?
Dave
On May 7, 2007, at 4:35 PM, David Liontooth wrote:
Great question. And even before the question of a local mirror: is there a format for packaging a set of binaries that can then be installed into an existing system -- like a *.deb or *.rpm file?
Why not just mount /opt from a master machine that build the archives? Assuming you don't run into architecture or filesystem issues, any reason why this wouldn't work? I should do this myself, actually. -- Paul Beard words: http://paulbeard.org/wordpress pictures: http://www.flickr.com/photos/pdb206/ Are you trying to win an argument or solve a problem?
On May 7, 2007, at 18:35, David Liontooth wrote:
Rick Gigger wrote:
Is it possible to have an internal macports mirror that also contains binaries, so I can compile all the ports I need once and install them on several boxes instead of re-compiling everything on every single box?
Great question. And even before the question of a local mirror: is there a format for packaging a set of binaries that can then be installed into an existing system -- like a *.deb or *.rpm file?
Such a format does exist: It's called a .pkg file and you interact with them every time you install an Apple software update. However, I don't think there's any functionality in MacPorts to create or use package files.
On May 7, 2007, at 17:39, Rick Gigger wrote:
Is it possible to have an internal macports mirror that also contains binaries, so I can compile all the ports I need once and install them on several boxes instead of re-compiling everything on every single box?
That functionality does not exist. One small thing you could do: after you install all the ports you want on one system, you can copy the /opt/local/var/db/dports/ distfiles directory from that machine to another machine where you want to install ports. That way the second machine will not need to download the distribution files again. However, it will still need to compile and install the software itself. You could attempt to copy other parts of /opt/local to the other machine as well. I don't know how well that would work. Certainly, the machines would have be set up virtually identically in other respects -- same processor architecture to be sure, same exact OS version, same OS updates installed, same X11. It is an eventual goal of MacPorts to provide binaries of the ports, rather than make everyone compile them themselves. However, I estimate we're still a long way away from anything resembling that kind of functionality.
On May 8, 2007, at 11:28 AM, Ryan Schmidt wrote:
On May 7, 2007, at 17:39, Rick Gigger wrote:
Is it possible to have an internal macports mirror that also contains binaries, so I can compile all the ports I need once and install them on several boxes instead of re-compiling everything on every single box?
That functionality does not exist.
One small thing you could do: after you install all the ports you want on one system, you can copy the /opt/local/var/db/dports/ distfiles directory from that machine to another machine where you want to install ports. That way the second machine will not need to download the distribution files again. However, it will still need to compile and install the software itself.
You could attempt to copy other parts of /opt/local to the other machine as well. I don't know how well that would work. Certainly, the machines would have be set up virtually identically in other respects -- same processor architecture to be sure, same exact OS version, same OS updates installed, same X11.
It is an eventual goal of MacPorts to provide binaries of the ports, rather than make everyone compile them themselves. However, I estimate we're still a long way away from anything resembling that kind of functionality.
I believe this functionality exists, on the contrary. It's called archive. You enable it in ports.conf: portarchivemode yes Then all compiled ports are stored in /opt/local/var/db/dports/ packages/darwin/{powerpc,intel}/ There is a limitation, though. The archives are matched against the architecture (powerpc/intel), but not against the version of the system. If you build powerpc archives on a 10.3.9, you should not copy them over to a 10.4.9 box. However, this functionality is broken in 1.4.3. This bug was fixed three weeks ago, but the people in charge here think we should not make too often releases, so you'll have to use trunk or wait for the fix. Paul
On May 7, 2007, at 8:04 PM, Paul Guyot wrote:
However, this functionality is broken in 1.4.3. This bug was fixed three weeks ago, but the people in charge here think we should not make too often releases, so you'll have to use trunk or wait for the fix.
I'm sure this has been discussed elsewhere but if this was a democracy, I would vote for more frequent/regular releases, especially if as alluded above, there is some breakage that can be fixed. I know we just went through a messy release of 1.4.[0123] but if the swelling has gone down from that one, when do you think the next release will be? -- Paul Beard words: http://paulbeard.org/wordpress pictures: http://www.flickr.com/photos/pdb206/ Are you trying to win an argument or solve a problem?
On May 7, 2007, at 8:15 PM, paul beard wrote:
On May 7, 2007, at 8:04 PM, Paul Guyot wrote:
However, this functionality is broken in 1.4.3. This bug was fixed three weeks ago, but the people in charge here think we should not make too often releases, so you'll have to use trunk or wait for the fix.
I'm sure this has been discussed elsewhere but if this was a democracy, I would vote for more frequent/regular releases, especially if as alluded above, there is some breakage that can be fixed. I know we just went through a messy release of 1.4.[0123] but if the swelling has gone down from that one, when do you think the next release will be?
I'm all for frequent releases, and have just been waiting for a bit of time in my schedule, and concurrence from others, to make a release. I don't think Paul is being quite fair. James
On May 7, 2007, at 7:23 PM, Ryan Schmidt wrote:
Such a format does exist: It's called a .pkg file and you interact with them every time you install an Apple software update. However, I don't think there's any functionality in MacPorts to create or use package files.
Oh really? root@sam-> port pkg zsh-devel ---> Fetching zsh-devel ---> Verifying checksum(s) for zsh-devel ---> Extracting zsh-devel ---> Configuring zsh-devel ---> Building zsh-devel with target all ---> Staging zsh-devel into destroot ---> Creating pkg for zsh-devel-4.3.4 root@sam-> ls -l ~/Src/macports/dports/shells/zsh-devel/work/ total 4 -rw-r--r-- 1 root admin 222 May 8 00:37 .darwinports.zsh- devel.state drwxrwxr-x 3 root wheel 102 May 8 00:36 destroot drwxr-xr-x 42 root admin 1428 May 8 00:35 zsh-4.3.4 drwxr-xr-x 3 root admin 102 May 8 00:36 zsh-devel-4.3.4.pkg ^^^^^^^^^^^^^^^ You can also do use the "mpkg" target for ports with dependencies; this will cause a metapackage to be built which contains all the deps, making the package stand-alone. This functionality has been in macports for a long time. :) - Jordan
On Mon, May 07, 2007 at 09:28:42PM -0500, Ryan Schmidt wrote:
On May 7, 2007, at 17:39, Rick Gigger wrote:
Is it possible to have an internal macports mirror that also contains binaries, so I can compile all the ports I need once and install them on several boxes instead of re-compiling everything on every single box?
That functionality does not exist.
One small thing you could do: after you install all the ports you want on one system, you can copy the /opt/local/var/db/dports/ distfiles directory from that machine to another machine where you want to install ports. That way the second machine will not need to download the distribution files again. However, it will still need to compile and install the software itself.
You could attempt to copy other parts of /opt/local to the other machine as well. I don't know how well that would work. Certainly, the machines would have be set up virtually identically in other respects -- same processor architecture to be sure, same exact OS version, same OS updates installed, same X11.
I routinely use `rsync' to synchronize the `/opt/local' tree between a G5 and a G4 ppc: `{sudo} rsync --delete -avW /opt/local/ {some_host_name}:/opt/local' (don't forget the trailing `/' in the first path. for testing, first add the no-op option `-n' to the rsync command and check the output). my approach is to install everything via `ports' only on the fast G5 and than to rysnc to the G4. works perfectly for me. the only thing I know of which does not reside in `/opt/local' relates to `Tcl': there are some things in '/Library/Tcl/darwinports1.0/' which should be synchronized in the same way between the two machines (but maybe this is obsolete?)
It is an eventual goal of MacPorts to provide binaries of the ports, rather than make everyone compile them themselves. However, I estimate we're still a long way away from anything resembling that kind of functionality.
_______________________________________________ macports-users mailing list macports-users@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo/macports-users
On May 8, 2007, at 12:17 PM, James Berry wrote:
However, this functionality is broken in 1.4.3. This bug was fixed three weeks ago, but the people in charge here think we should not make too often releases, so you'll have to use trunk or wait for the fix.
I'm sure this has been discussed elsewhere but if this was a democracy, I would vote for more frequent/regular releases, especially if as alluded above, there is some breakage that can be fixed. I know we just went through a messy release of 1.4.[0123] but if the swelling has gone down from that one, when do you think the next release will be?
I'm all for frequent releases, and have just been waiting for a bit of time in my schedule, and concurrence from others, to make a release. I don't think Paul is being quite fair.
I do not think that you should do releases more frequently, because I know this takes a lot of time. I was not criticizing the fact that you did not release 1.4.40 three weeks ago when the fix was committed. Our infrastructure (language, user base, supported operating systems, developers) is such that having very frequent releases is the best policy, something I think you agree with, but jmpp@ disagrees with. Yet, you, jmpp@ or whoever who could do releases are way too busy to do them. And if we are not busy, it's a waste of time compared to other MacPorts-related tasks, such as mwpa for example. Consequently, I believe that we simply should not have releases at all, until we have some form of continuous integration. Paul -- http://paul-guyot.com/
On May 8, 2007, at 03:40, Joerg van den Hoff wrote:
the only thing I know of which does not reside in `/opt/local' relates to `Tcl': there are some things in '/Library/Tcl/darwinports1.0/' which should be synchronized in the same way between the two machines (but maybe this is obsolete?)
No, that's still current and is required by the base MacPorts system. It won't be updated, though, unless you upgrade to a newer version of MacPorts.
On 5/8/07, Jordan K. Hubbard <jkh@brierdr.com> wrote:
You can also do use the "mpkg" target for ports with dependencies; this will cause a metapackage to be built which contains all the deps, making the package stand-alone.
I just tried that. It seems not to work quite as anticipated. $ sudo port mpkg gnucash +guile16 ... results in port packaging guile instead of guile16, which makes the resulting package useless. It seems as if port mpkg follows the original dependencies specified in the port, not those in the variant given, nor even those in the default variant. Any hints how to fix this? This functionality would probably be useful to some people who don't really want to install MacPorts at all, but just need that one package. Or is there anything else I'm doing wrong? Regards, Marc PS: If you try the command above, be prepared for a long wait. On my old iBook G4, after packaging gnucash itself, port took 15 minutes just to compute dependencies before beginning to collect them.
On May 8, 2007, at 5:48 AM, Marc André Selig wrote:
I just tried that. It seems not to work quite as anticipated.
$ sudo port mpkg gnucash +guile16
... results in port packaging guile instead of guile16, which makes the resulting package useless.
It seems as if port mpkg follows the original dependencies specified in the port, not those in the variant given, nor even those in the default variant.
I believe you have indeed found a bug! The pkg / mpkg creation code was written a long time ago and I would not be at all surprised to learn that it has not kept pace with other changes in macports. The code for all the packaging procedures lives in macports/base/src/ package1.0/ and represent an excellent learning opportunity / potential contribution for anyone. :-) - Jordan
Jordan K. Hubbard wrote:
On May 8, 2007, at 5:48 AM, Marc André Selig wrote:
I just tried that. It seems not to work quite as anticipated.
$ sudo port mpkg gnucash +guile16
... results in port packaging guile instead of guile16, which makes the resulting package useless.
It seems as if port mpkg follows the original dependencies specified in the port, not those in the variant given, nor even those in the default variant.
I believe you have indeed found a bug! The pkg / mpkg creation code was written a long time ago and I would not be at all surprised to learn that it has not kept pace with other changes in macports. The code for all the packaging procedures lives in macports/base/src/package1.0/ and represent an excellent learning opportunity / potential contribution for anyone. :-)
So it is possible to create packages with macports but right now there is a bug that is preventing it from correctly tracking dependencies. Is that correct? But once that is fixed I should be able to specify all of the ports that I want installed and have pgk and mpkg files created so that I could just mount an image with these package files on it and install binaries. Is that right?
On May 14, 2007, at 2:07 PM, Rick Gigger wrote:
So it is possible to create packages with macports but right now there is a bug that is preventing it from correctly tracking dependencies. Is that correct?
But once that is fixed I should be able to specify all of the ports that I want installed and have pgk and mpkg files created so that I could just mount an image with these package files on it and install binaries. Is that right?
Right. The only question is: Who will fix the bug? :-) - Jordan
On May 8, 2007, at 00:38, Jordan K. Hubbard wrote:
On May 7, 2007, at 7:23 PM, Ryan Schmidt wrote:
Such a format does exist: It's called a .pkg file and you interact with them every time you install an Apple software update. However, I don't think there's any functionality in MacPorts to create or use package files.
Oh really?
[snip]
You can also do use the "mpkg" target for ports with dependencies; this will cause a metapackage to be built which contains all the deps, making the package stand-alone.
This functionality has been in macports for a long time. :)
And dpkg (and rpm) too! There's even a particularly frightening script I wrote to build dpkg apt-get repositories from a list of ports in base/portmgr/dpkgall.tcl Of course, xar is the better choice, now. Freak out! landonf@timor:dict> sudo port dpkg dict [snip] ---> Creating dpkg for dict-1.9.7 Of course, it seems to be slightly broken, now: landonf@timor:macports/dports/textproc/dict> sudo dpkg --force-all -- install work/dict_1.9.7_darwin-i386.deb dpkg - warning, overriding problem because --force enabled: package architecture (darwin-i386) does not match system (i686-darwin) Selecting previously deselected package dict. (Reading database ... 0 files and directories currently installed.) Unpacking dict (from .../dict_1.9.7_darwin-i386.deb) ... dpkg: dict: dependency problems, but configuring anyway as you request: dict depends on libtool (>= 1.5.22); however: Package libtool is not installed. Setting up dict (1.9.7) ... landonf@timor:macports/dports/textproc/dict> dpkg --list Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Installed/Config-files/Unpacked/Failed-config/Half- installed |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad) ||/ Name Version Description +++-=================================================- =================================================- ======================================================================== ========================================== ii dict 1.9.7 Dictionary Server Protocol (RFC2229) client
participants (10)
-
David Liontooth
-
James Berry
-
Joerg van den Hoff
-
Jordan K. Hubbard
-
Landon Fuller
-
Marc André Selig
-
paul beard
-
Paul Guyot
-
Rick Gigger
-
Ryan Schmidt