Ports info SQL database feeder (Fwd: [27415] trunk/base/portmgr/packaging)
Evening everyone! Per the message below, in r27415 I committed a rather big facelift to the PortIndex2MySQL script that basically gives us an updated tool that extracts important information out of every single port in our tree (per a valid entry in sources.conf) and injects it as SQL statements into any (I think) SQL based db of our choosing, first printing them to a temporary flat file. I haven't tested database functionality yet as I've been struggling to find some time to setup MySQL5 locally, but for the rest I can account for 100% functionality. I would love it if some of our database savvy members (Ryan? Chris? James? Anyone else?) could take the time to look at the script and find interesting ways to improve and extend it; for instance, what other tables could we create with valuable information out of our ports? One obvious thing we need to think about is, of course, what to do with the script's results. That is to say, we need to put this baby to good use! The old ports.php page (trunk/www/ports.php, OpenDarwin days) was, I believe, a good database client, so I'll work on it next to bring it up to par with the new PortIndex2MySQL script. But other than that, it's no secret that our current webpage is a sorry excuse for an information portal and working on ports.php without knowing if it can be integrated into a reworked site (Kevin?) might be a bit of a waste of time. I personally would love to "donate" my time to such cause if it helps us get on track to start working on a revamped site -- and I'm positive a lively updated page with ports information is a corner stone of any new design! There's also the issue of James' http://db.macports.org portal for mpwa, which is written in Ruby on Rails. Are there any plans to use mpwa and autosubmit (trunk/base/portmgr/autosubmit.tcl) to improve our web presence? Could such portal benefit from the results of PortIndex2MySQL? I would hate to see our little team waste its time working on two different approaches (php & RoR) to fix the same problem: a page with always up to date information of our ports, a la ports.php. So people, feedback please! Running PortIndex2MySQL is now as simple as creating a cron/launchd entry for it as it stands, but we still need to put the results to good use. We most definitely need a completely revamped web presence and I would love us to start working on it sooner than later (lets admit it, there's a disgustingly obvious reason why darwinports.com is so successful, as much as we may hate it!). I can interface with Mac OS Forge for resources and setup (there's also the issue of integrating the new guide), but we first need to agree on what we want to see in a new site and how to implement it. Regards,... -jmpp Begin forwarded message:
From: source_changes@macosforge.org Date: August 2, 2007 11:15:12 PM GMT-04:00 To: macports-changes@lists.macosforge.org Subject: [27415] trunk/base/portmgr/packaging Reply-To: macports-dev@lists.macosforge.org
Revision 27415
Author jmpp@macports.org
Date 2007-08-02 20:15:11 -0700 (Thu, 02 Aug 2007)
Log Message: Sizeable facelift to the PortIndex2MySQK.tcl script. Highlights: * Bring it up to date wrt the macports1.0 api and into the new namespace; * Make it self-contained, writing its sql output to a flat file directly and then piping its contents to a proper database command (like mysql5(1)); this eliminates both the need to load the mysqltcl library or, the other case, wrap the tcl code with a shell script to do all the needed redirection to get the information into the db; * Get rid of a lot of now unnecessary code due to the rework above; * Provide abstraction variables to easily adapt the script to any scenario and a proc to read the db password from a file that must exist (this bit could definitely use some improvements, like for passwordless db's); * Add build and run dependencies for a given port as sql statements into the db (previously only lib deps were being recorded); * For fields with multiple and hierarchycal values, like categories, index them sequentially up from 1 (1 being topmost), rather than 1 for the first one (most important) and 0 for the rest (flat second place) as it was being done; * Use procs in the macports1.0 package, like ui_error; * Provide a Makefile (currently unhooked from MacPorts build system) to tie the script into an already configured MacPorts installation. So, in a nutshell, the script now works to generate valid SQL statements for our ports tree, we can extend it to do anything else we want and then find a client that will read such db and thus put it to good use ;-) (old ports.php page, here I come!)
Modified Paths trunk/base/portmgr/packaging/PortIndex2MySQL.tcl
Added Paths trunk/base/portmgr/packaging/Makefile
We can host the ports page on Mac OS Forge. On Aug 5, 2007, at 10:45 PM, Juan Manuel Palacios wrote:
Evening everyone!
Per the message below, in r27415 I committed a rather big facelift to the PortIndex2MySQL script that basically gives us an updated tool that extracts important information out of every single port in our tree (per a valid entry in sources.conf) and injects it as SQL statements into any (I think) SQL based db of our choosing, first printing them to a temporary flat file. I haven't tested database functionality yet as I've been struggling to find some time to setup MySQL5 locally, but for the rest I can account for 100% functionality.
I would love it if some of our database savvy members (Ryan? Chris? James? Anyone else?) could take the time to look at the script and find interesting ways to improve and extend it; for instance, what other tables could we create with valuable information out of our ports? One obvious thing we need to think about is, of course, what to do with the script's results. That is to say, we need to put this baby to good use! The old ports.php page (trunk/www/ports.php, OpenDarwin days) was, I believe, a good database client, so I'll work on it next to bring it up to par with the new PortIndex2MySQL script. But other than that, it's no secret that our current webpage is a sorry excuse for an information portal and working on ports.php without knowing if it can be integrated into a reworked site (Kevin?) might be a bit of a waste of time. I personally would love to "donate" my time to such cause if it helps us get on track to start working on a revamped site -- and I'm positive a lively updated page with ports information is a corner stone of any new design!
There's also the issue of James' http://db.macports.org portal for mpwa, which is written in Ruby on Rails. Are there any plans to use mpwa and autosubmit (trunk/base/portmgr/autosubmit.tcl) to improve our web presence? Could such portal benefit from the results of PortIndex2MySQL? I would hate to see our little team waste its time working on two different approaches (php & RoR) to fix the same problem: a page with always up to date information of our ports, a la ports.php.
So people, feedback please! Running PortIndex2MySQL is now as simple as creating a cron/launchd entry for it as it stands, but we still need to put the results to good use. We most definitely need a completely revamped web presence and I would love us to start working on it sooner than later (lets admit it, there's a disgustingly obvious reason why darwinports.com is so successful, as much as we may hate it!). I can interface with Mac OS Forge for resources and setup (there's also the issue of integrating the new guide), but we first need to agree on what we want to see in a new site and how to implement it.
Regards,...
-jmpp
Begin forwarded message:
From: source_changes@macosforge.org Date: August 2, 2007 11:15:12 PM GMT-04:00 To: macports-changes@lists.macosforge.org Subject: [27415] trunk/base/portmgr/packaging Reply-To: macports-dev@lists.macosforge.org
Revision 27415
Author jmpp@macports.org
Date 2007-08-02 20:15:11 -0700 (Thu, 02 Aug 2007)
Log Message: Sizeable facelift to the PortIndex2MySQK.tcl script. Highlights: * Bring it up to date wrt the macports1.0 api and into the new namespace; * Make it self-contained, writing its sql output to a flat file directly and then piping its contents to a proper database command (like mysql5(1)); this eliminates both the need to load the mysqltcl library or, the other case, wrap the tcl code with a shell script to do all the needed redirection to get the information into the db; * Get rid of a lot of now unnecessary code due to the rework above; * Provide abstraction variables to easily adapt the script to any scenario and a proc to read the db password from a file that must exist (this bit could definitely use some improvements, like for passwordless db's); * Add build and run dependencies for a given port as sql statements into the db (previously only lib deps were being recorded); * For fields with multiple and hierarchycal values, like categories, index them sequentially up from 1 (1 being topmost), rather than 1 for the first one (most important) and 0 for the rest (flat second place) as it was being done; * Use procs in the macports1.0 package, like ui_error; * Provide a Makefile (currently unhooked from MacPorts build system) to tie the script into an already configured MacPorts installation. So, in a nutshell, the script now works to generate valid SQL statements for our ports tree, we can extend it to do anything else we want and then find a client that will read such db and thus put it to good use ;-) (old ports.php page, here I come!)
Modified Paths trunk/base/portmgr/packaging/PortIndex2MySQL.tcl
Added Paths trunk/base/portmgr/packaging/Makefile
participants (2)
-
Juan Manuel Palacios
-
Kevin Van Vechten