#51689: preserve_runtime_libraries convenience PortGroup --------------------------+-------------------------------- Reporter: rjvbertin@… | Owner: macports-tickets@… Type: submission | Status: new Priority: Normal | Milestone: Component: contrib | Version: Resolution: | Keywords: Port: | --------------------------+-------------------------------- Old description:

This is mostly for reference though I'd love to see a similar feature in MacPorts:

Many if not most Linux distributions have one way or another to keep older versions of shared libraries installed if the newer versions introduce ABI incompatibilities. Advantages are less and typically smaller upgrades for the users, less stress on the build bots and as a result, a more reliable upgrade process. That latter claim may need explanation (and may not be totally relevant for MacPorts): building binary packages takes time, and while those builds are in process users starting an upgrade may find themselves in a situation where the new packages aren't yet available. In extreme cases that may lead to uninstallation of numerous packages for which dependency criteria are no longer met. I've experienced that personally with Kubuntu during a KDE version update; I was lucky to have taken a ZFS snapshot before starting the update.

To return closer to home: I was still running a 0.4 port:webp and thought I'd update it since it's could be a dependency for QtWebEngine (if that package allowed to build against the system webp on OS X). Webp 0.5 changes the library versioning, so a number of ports broke, including a few that are quite (very) expensive to build:

``` ---> Found 11 broken files, matching files to ports ---> Found 7 broken ports: graphviz @2.38.0 +pangocairo+universal+x11 /opt/local/bin/dot_builtins /opt/local/lib/graphviz/libgvplugin_webp.6.dylib ImageMagick @6.9.0-0 +x11 /opt/local/lib/ImageMagick-6.9.0/modules-Q16/coders/webp.so kde4-runtime @4.16.03.80.6 +docs /opt/local/lib/kde4/plugins/imageformats/kimg_webp.so opencv @3.1.0 +contrib+eigen+python27+qt5+tbb /opt/local/lib/libopencv_imgcodecs.3.1.0.dylib webkit-gtk @2.4.7 +video /opt/local/lib/libwebkitgtk-1.0.0.dylib webkit-gtk3 @2.4.7 +video /opt/local/lib/libwebkitgtk-3.0.0.dylib qt5-kde-devel @5.6.1 +harfbuzz+langselect+qt5kde /opt/local/libexec/qt5/Library/Frameworks/QtWebKit.framework/Versions/5/QtWebKit /opt/local/share/qt5/plugins/imageformats/libqwebp.dylib ```

The attached PortGroup provides an easy first-approach/PoC solution to this issue: it copies the specified runtime libraries from ${prefix} to ${destroot} during the post-destroot, if they do not already exist in the destroot.

For instance, in a personal port:webp I used for testing I only needed to include the PortGroup and then add the line

``` preserve_libraries [glob -nocomplain ${prefix}/lib/libwebp*.*.dylib] ```

This simply allows currently installed ports to remain functional using their current dependencies and alleviates the need to revbump all dependents. They'll just move to using the new dependency version when they're rebuilt for other reasons (upgrade, reinstall, ...). I'm aware that this kind of practice may lead to runtime weirdness esp. with C++ libraries if different versions cannot coexist in a single application's runtime image. Easy enough to avoid and apart from that I see only one true issue: it won't be trivial to limit the number of old versions that will be preserved via a generic mechanism.

New description: This is mostly for reference though I'd love to see a similar feature in MacPorts: Many if not most Linux distributions have one way or another to keep older versions of shared libraries installed if the newer versions introduce ABI incompatibilities. Advantages are less and typically smaller upgrades for the users, less stress on the build bots and as a result, a more reliable upgrade process. That latter claim may need explanation (and may not be totally relevant for MacPorts): building binary packages takes time, and while those builds are in process users starting an upgrade may find themselves in a situation where the new packages aren't yet available. In extreme cases that may lead to uninstallation of numerous packages for which dependency criteria are no longer met. I've experienced that personally with Kubuntu during a KDE version update; I was lucky to have taken a ZFS snapshot before starting the update. To return closer to home: I was still running a 0.4 port:webp and thought I'd update it since it's could be a dependency for QtWebEngine (if that package allowed to build against the system webp on OS X). Webp 0.5 changes the library versioning, so a number of ports broke, including a few that are quite (very) expensive to build: {{{ ---> Found 11 broken files, matching files to ports ---> Found 7 broken ports: graphviz @2.38.0 +pangocairo+universal+x11 /opt/local/bin/dot_builtins /opt/local/lib/graphviz/libgvplugin_webp.6.dylib ImageMagick @6.9.0-0 +x11 /opt/local/lib/ImageMagick-6.9.0/modules-Q16/coders/webp.so kde4-runtime @4.16.03.80.6 +docs /opt/local/lib/kde4/plugins/imageformats/kimg_webp.so opencv @3.1.0 +contrib+eigen+python27+qt5+tbb /opt/local/lib/libopencv_imgcodecs.3.1.0.dylib webkit-gtk @2.4.7 +video /opt/local/lib/libwebkitgtk-1.0.0.dylib webkit-gtk3 @2.4.7 +video /opt/local/lib/libwebkitgtk-3.0.0.dylib qt5-kde-devel @5.6.1 +harfbuzz+langselect+qt5kde /opt/local/libexec/qt5/Library/Frameworks/QtWebKit.framework/Versions/5/QtWebKit /opt/local/share/qt5/plugins/imageformats/libqwebp.dylib }}} The attached PortGroup provides an easy first-approach/PoC solution to this issue: it copies the specified runtime libraries from ${prefix} to ${destroot} during the post-destroot, if they do not already exist in the destroot. For instance, in a personal port:webp I used for testing I only needed to include the PortGroup and then add the line {{{ preserve_libraries [glob -nocomplain ${prefix}/lib/libwebp*.*.dylib] }}} This simply allows currently installed ports to remain functional using their current dependencies and alleviates the need to revbump all dependents. They'll just move to using the new dependency version when they're rebuilt for other reasons (upgrade, reinstall, ...). I'm aware that this kind of practice may lead to runtime weirdness esp. with C++ libraries if different versions cannot coexist in a single application's runtime image. Easy enough to avoid and apart from that I see only one true issue: it won't be trivial to limit the number of old versions that will be preserved via a generic mechanism. -- Comment (by ryandesign@…): This won't work with binaries built by our buildbot system, because no ports are active when a port build is started on our buildbot system. More generally, it conflicts with our goal of ReproducibleBuilds. So I don't think we want this solution for this issue. You should bring the matter up on the macports-devel mailing list for discussion of the problem and possible solutions before implementing a particular solution. It also seems like you're looking in part for a solution to a problem you caused for yourself. Regarding your example of webp, it was updated to version 0.5 in r144197 back in January 2016, and all the ports you show as broken were revbumped at the same time. Our build server will have built binaries of those packages at that time, if possible, such that you should now be able to upgrade those ports and receive those binaries without having to wait for compilation. kde4-runtime appears to be an exception in your case because you're running a locally-modified newer version than the one we have in MacPorts, so yes you'll have to recompile that, as well as other ports for which binaries are not available, such as ImageMagick. -- Ticket URL: <https://trac.macports.org/ticket/51689#comment:1> MacPorts <https://www.macports.org/> Ports system for OS X

#51689: preserve_runtime_libraries convenience PortGroup --------------------------+-------------------------------- Reporter: rjvbertin@… | Owner: macports-tickets@… Type: submission | Status: closed Priority: Normal | Milestone: Component: contrib | Version: Resolution: wontfix | Keywords: Port: | --------------------------+-------------------------------- Changes (by ryandesign@…): * status: new => closed * resolution: => wontfix Comment: We've already solved this problem: whenever a port is updated in such a way that one of its libraries changes its install_name (including changing its major version), the person who updates that port shall also revbump all ports linking with that library so that they get rebuilt. Yes, binary packages may take some time for our buildbot to compile, or may not be available in all situations so users may need to compile the upgrades on their own systems. I realize you don't like this solution, but it's the one we've chosen for now. You're welcome to discuss it on the mailing list, but your proposed portgroup in this ticket is unlikely to be an acceptable solution. -- Ticket URL: <https://trac.macports.org/ticket/51689#comment:3> MacPorts <https://www.macports.org/> Ports system for macOS

#51689: preserve_runtime_libraries convenience PortGroup --------------------------+-------------------------------- Reporter: rjvbertin@… | Owner: macports-tickets@… Type: submission | Status: closed Priority: Normal | Milestone: Component: contrib | Version: Resolution: wontfix | Keywords: Port: | --------------------------+-------------------------------- Comment (by rjvbertin@…): In case anyone is interested: I've made this feature work a bit more cleanly but storing the legacy runtimes in a dedicated repository. That requires additional symlinks, but it does make it immediately clear which runtimes are potentially obsolete, whom they belong to - and testing whether they're still used is as easy as moving the symlink aside before running `port -v rev-upgrade`. -- Ticket URL: <https://trac.macports.org/ticket/51689#comment:5> MacPorts <https://www.macports.org/> Ports system for the Mac operating system