Simplify sourceforge fetch group

Ryan Schmidt ryandesign at macports.org
Mon Jun 13 11:57:00 PDT 2011


On May 23, 2011, at 14:00, Joshua Root wrote:

> The original reason (r21768) for having more than just the redirector
> was in case it goes down. The main downside to removing the rest now is
> that the ping time for the redirector is not the same as that of the
> mirror it redirects to, but it will be compared to those of the MacPorts
> mirrors during sorting.

I don't think we have the ping time advantage now anyway. I think the vast majority of our ports that use SourceForge for downloads assume the old "flat" namespace of files, for example:

$ port distfiles expat
--->  Distfiles for expat
[expat-2.0.1.tar.gz] /opt/local/var/macports/distfiles/expat/expat-2.0.1.tar.gz
 md5: ee8b492592568805593f81f8cdf2a04c
 sha1: 663548c37b996082db1f2f2c32af060d7aa15c2d
 rmd160: d31bcb152adaff9e358968be1ea901e1b4ed3b2f
  http://downloads.sourceforge.net/expat/expat-2.0.1.tar.gz
  http://aarnet.dl.sourceforge.net/expat/expat-2.0.1.tar.gz
  http://biznetnetworks.dl.sourceforge.net/expat/expat-2.0.1.tar.gz
  http://cdnetworks-us-1.dl.sourceforge.net/expat/expat-2.0.1.tar.gz
  http://cdnetworks-us-2.dl.sourceforge.net/expat/expat-2.0.1.tar.gz
  http://cdnetworks-kr-1.dl.sourceforge.net/expat/expat-2.0.1.tar.gz
  http://cdnetworks-kr-2.dl.sourceforge.net/expat/expat-2.0.1.tar.gz

[snip]

However, SourceForge reorganized their files into folders some time ago. And requesting any of the URLs above results in multiple redirects: first back to the main SourceForge redirector, and then on to a mirror of its choosing:

$ curl -L -I http://cdnetworks-us-1.dl.sourceforge.net/expat/expat-2.0.1.tar.gz
HTTP/1.1 302 Found
Date: Mon, 13 Jun 2011 18:52:14 GMT
Server: Apache
Location: http://downloads.sourceforge.net/expat/expat-2.0.1.tar.gz?download&failedmirror=cdnetworks-us-1.dl.sourceforge.net
Connection: close
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 301 Moved Permanently
Access-Control-Allow-Origin: *
X-Powered-By: PHP/5.2.9
Location: http://downloads.sourceforge.net/project/expat/expat/2.0.1/expat-2.0.1.tar.gz?download=&failedmirror=cdnetworks-us-1.dl.sourceforge.net
Content-type: text/html
Date: Mon, 13 Jun 2011 18:52:15 GMT
Server: lighttpd/1.4.26

HTTP/1.1 302 Found
Access-Control-Allow-Origin: *
X-Powered-By: PHP/5.2.9
Content-Disposition: attachment; filename="expat-2.0.1.tar.gz"
Location: http://softlayer.dl.sourceforge.net/project/expat/expat/2.0.1/expat-2.0.1.tar.gz
Content-type: text/html
Date: Mon, 13 Jun 2011 18:52:15 GMT
Server: lighttpd/1.4.26

HTTP/1.1 200 OK
Date: Mon, 13 Jun 2011 18:52:15 GMT
Server: Apache
Last-Modified: Thu, 07 Jun 2007 02:37:51 GMT
ETag: "25e240f7-6cff8-43247cd0b9dc0"
Accept-Ranges: bytes
Content-Length: 446456
Connection: close
Content-Type: application/x-gzip

So all that our list of mirrors is doing is introducing one extra redirect (if the file is found), or dozens more redirects and DNS lookups (if the file is not found on any SourceForge server). And sometimes causing users checksum errors when mirrors are taken offline (see #29810).





More information about the macports-dev mailing list