[MacPorts] #14891: PATCH: use the fastest mirror in fetch phase
#14891: PATCH: use the fastest mirror in fetch phase ------------------------------+--------------------------------------------- Reporter: jmr@macports.org | Owner: macports-tickets@lists.macosforge.org Type: enhancement | Status: new Priority: Normal | Milestone: MacPorts base enhancements Component: base | Version: 1.7.0 Keywords: | ------------------------------+--------------------------------------------- Here's a patch which makes the fetch phase ping each of the candidate download sites, and then sort the list in ascending order of ping time. This is my first attempt at writing Tcl, so no doubt it could be done better, and I welcome any advice. Some issues that need to be thought about: * We probably want to special-case some mirrors. We always want to try MacPorts svn last, and the primary site for each mirror group should be second-last. * Caching. Is it worth the effort? Maybe pinging every mirror every time is fine. When do we refresh cache entries? * User hints/overrides. If the user tells us that certain mirrors are good, we should always try them first when eligible. -- Ticket URL: <http://trac.macosforge.org/projects/macports/ticket/14891> MacPorts </projects/macports> Ports system for Mac OS
#14891: PATCH: use the fastest mirror in fetch phase -------------------------------+-------------------------------------------- Reporter: jmr@macports.org | Owner: macports-tickets@lists.macosforge.org Type: enhancement | Status: new Priority: Normal | Milestone: MacPorts base enhancements Component: base | Version: 1.7.0 Resolution: | Keywords: -------------------------------+-------------------------------------------- Comment (by wsiegrist@apple.com): I tested with and without the patch on 2 random ports: less and php5. less: the svn.macosforge.org sites won and were tried first, even though they dont have the files. You should probably exclude macosforge and macports sites from the ping test until we have mirroring going. php5: it took longer to fetch because it waited up to 5s for each mirror it seems. Maybe try to find a winner with "-t1", then "-t5" to minimize the impact of downed mirrors. Overall I like the idea though. -- Ticket URL: <http://trac.macosforge.org/projects/macports/ticket/14891#comment:1> MacPorts </projects/macports> Ports system for Mac OS
#14891: PATCH: use the fastest mirror in fetch phase -------------------------------+-------------------------------------------- Reporter: jmr@macports.org | Owner: macports-tickets@lists.macosforge.org Type: enhancement | Status: new Priority: Normal | Milestone: MacPorts base enhancements Component: base | Version: 1.7.0 Resolution: | Keywords: -------------------------------+-------------------------------------------- Comment (by jmr@macports.org): Updated the patch to try svn.macports.org last, ping with only one packet, and use a 1-second timeout in the interest of speed. Slow mirrors won't be sorted in quite the correct order, but I don't think that that's a real problem. Working mirrors that just don't respond to ping will go last in any case, but again, things should still generally be better with the patch than without. It would be ideal to run all the pings in parallel, but I don't know how to do that. -- Ticket URL: <http://trac.macosforge.org/projects/macports/ticket/14891#comment:2> MacPorts </projects/macports> Ports system for Mac OS
#14891: PATCH: use the fastest mirror in fetch phase -------------------------------+-------------------------------------------- Reporter: jmr@macports.org | Owner: macports-tickets@lists.macosforge.org Type: enhancement | Status: new Priority: Normal | Milestone: MacPorts base enhancements Component: base | Version: 1.7.0 Resolution: | Keywords: -------------------------------+-------------------------------------------- Comment (by raimue@macports.org): Added 'sortsites2.diff‘; an updated version of the patch which avoids hardcoding 'svn.macports.org', but uses the fallback_mirror_list where this host actually comes from. -- Ticket URL: <http://trac.macosforge.org/projects/macports/ticket/14891#comment:3> MacPorts </projects/macports> Ports system for Mac OS
#14891: PATCH: use the fastest mirror in fetch phase -------------------------------+-------------------------------------------- Reporter: jmr@macports.org | Owner: macports-tickets@lists.macosforge.org Type: enhancement | Status: new Priority: Normal | Milestone: MacPorts base enhancements Component: base | Version: 1.7.0 Resolution: | Keywords: -------------------------------+-------------------------------------------- Comment (by jmr@macports.org): Updated: don't re-ping hosts that have already been pinged. Saves a lot of time in cases where hosts appear in the list several times. -- Ticket URL: <http://trac.macosforge.org/projects/macports/ticket/14891#comment:4> MacPorts </projects/macports> Ports system for Mac OS
#14891: PATCH: use the fastest mirror in fetch phase -------------------------------+-------------------------------------------- Reporter: jmr@macports.org | Owner: macports-tickets@lists.macosforge.org Type: enhancement | Status: new Priority: Normal | Milestone: MacPorts base enhancements Component: base | Version: 1.7.0 Resolution: | Keywords: -------------------------------+-------------------------------------------- Comment (by jmr@macports.org): Updated again: ping all hosts in parallel. -- Ticket URL: <http://trac.macosforge.org/projects/macports/ticket/14891#comment:5> MacPorts </projects/macports> Ports system for Mac OS
#14891: PATCH: use the fastest mirror in fetch phase -------------------------------+-------------------------------------------- Reporter: jmr@macports.org | Owner: macports-tickets@lists.macosforge.org Type: enhancement | Status: new Priority: Normal | Milestone: MacPorts base enhancements Component: base | Version: 1.7.0 Resolution: | Keywords: -------------------------------+-------------------------------------------- Comment (by wsiegrist@apple.com): Latest patch works well for me. -- Ticket URL: <http://trac.macosforge.org/projects/macports/ticket/14891#comment:6> MacPorts </projects/macports> Ports system for Mac OS
#14891: PATCH: use the fastest mirror in fetch phase -------------------------------+-------------------------------------------- Reporter: jmr@macports.org | Owner: macports-tickets@lists.macosforge.org Type: enhancement | Status: new Priority: Normal | Milestone: MacPorts base enhancements Component: base | Version: 1.7.0 Resolution: | Keywords: -------------------------------+-------------------------------------------- Comment (by jmr@macports.org): Updated: Recognise invalid hostnames like `image_magick.veidrodis.com`. (Sigh.) Check that the output from ping can be parsed as a number. Switched to using 3 ping packets and a 3-second timeout, since we effectively only have to wait for it once now that the pings run in parallel. Also simplified the code a bit. -- Ticket URL: <http://trac.macosforge.org/projects/macports/ticket/14891#comment:7> MacPorts </projects/macports> Ports system for Mac OS
#14891: PATCH: use the fastest mirror in fetch phase -------------------------------+-------------------------------------------- Reporter: jmr@macports.org | Owner: macports-tickets@lists.macosforge.org Type: enhancement | Status: new Priority: Normal | Milestone: MacPorts base enhancements Component: base | Version: 1.7.0 Resolution: | Keywords: -------------------------------+-------------------------------------------- Comment (by raimue@macports.org): Updated `sortsites4.diff`: If there is only one host (besides the fallbacks), don't ping or sort. -- Ticket URL: <http://trac.macosforge.org/projects/macports/ticket/14891#comment:8> MacPorts </projects/macports> Ports system for Mac OS
#14891: PATCH: use the fastest mirror in fetch phase -------------------------------+-------------------------------------------- Reporter: jmr@macports.org | Owner: macports-tickets@lists.macosforge.org Type: enhancement | Status: new Priority: Normal | Milestone: MacPorts base enhancements Component: base | Version: 1.7.0 Resolution: | Keywords: -------------------------------+-------------------------------------------- Changes (by raimue@macports.org): * cc: raimue@macports.org (added) Comment: This patch works just great. I encourage you to commit it, jmr! -- Ticket URL: <http://trac.macosforge.org/projects/macports/ticket/14891#comment:9> MacPorts </projects/macports> Ports system for Mac OS
#14891: PATCH: use the fastest mirror in fetch phase -------------------------------+-------------------------------------------- Reporter: jmr@macports.org | Owner: macports-tickets@lists.macosforge.org Type: enhancement | Status: closed Priority: Normal | Milestone: MacPorts base enhancements Component: base | Version: 1.7.0 Resolution: fixed | Keywords: -------------------------------+-------------------------------------------- Changes (by jmr@macports.org): * status: new => closed * resolution: => fixed Comment: Committed in r35748. Thanks for the help! -- Ticket URL: <http://trac.macosforge.org/projects/macports/ticket/14891#comment:10> MacPorts </projects/macports> Ports system for Mac OS
participants (1)
-
MacPorts