Hello Chris! So good to hear about your progress on the revamped registry, kudos! Some comments following: On Jul 28, 2007, at 5:28 PM, Chris Pickel wrote:

So, here's the deal:

As (a couple of) you are aware, my project is to build a more advanced registry; to keep more state for installed ports and in a more structured fashion.

I have written a pretty much feature-complete sqlite3-based replacement for receipts and the filemap at this point. There is a C library that's compiled as a static lib (cregistry), which is later compiled into a Tcl module (registry2.0). There's a test case; errors are thrown in such a way that they work with try/catch.

I've also written a script to up-convert a user's existing receipts/ filemap into the new format. It's reasonably fast (and was not so before I got smarter with sqlite transactions).

How would that script be run? Each user manually at their own discretion? Or through some upgrade glue internal to MacPorts? I would very much prefer the latter approach, something like my "upgrade" target in base/Makefile.in (later on I'll remove all that code once we consider enough time has gone by to upgrade most of our user base to the macports namespace, in the mean time I'm sure we can share that target for other upgrade purposes).

The Tcl code that makes *use* of the registry has not yet been modified. So, I could commit registry2 now without screwing anything up

Not sure what you mean by this. What hasn't been modified and where? You mean that if you commit registry2.0 now, trunk would still be using the old registry & filemap until you "hook up" your code?

but I have two concerns:

* Doing so would commit my local history (12 commits, I think). CIA has the annoying habit of squaring the number of commit messages sent out in sequence, and I don't want to flood #macports with 144 commit messages.

I would not worry one bit about this, message duplication is due to a misconfiguration on our side and we can always silence the CIA bot in the channel. Just let me know when you plan on committing and I'll take care of it.

* I don't feel there's a need to include it in 1.5.1. If someone wanted to go ahead and tag 1.5.1, that'd be great :)

I'm working my way toward 1.5.1, hopefully soonish now (targeting mid next week). But, likewise, nothing to worry about here, I'm working with the sweet svnmerge.py on the 1.5 branch so I can cherry pick what we merge in from turnk and what stays out. Also, the release tags are created off release branches, not trunk, so that's more leeway there.

I'm still working on three things before I move on to the dependency graph:

* I'm probably leaking memory like crazy. I'm currently trying to clamp down on this with `gdb` and `leaks`, but I've no previous experience with the latter. If anyone has suggestions in this category, I'd be happy to hear them (Paul mentioned TCL_MEM_DEBUG).

* Improving its speed, particularly as pertains to Eugene's usage of cregistry.

* Documentation. cregistry is largely documented, in the doxygen style, but I'm not sure of the best way to document registry2.0. I could do a manpage (but what to call it?), or more doxygen-style (but tools would confuse the C parameters of the actual functions with the Tcl parameters I'd document).

Registry2.0 should not have a man page for itself, those are mostly for higher level stuff like the port(1) utility and Portfile writing facilities. Our code should all be documented in a doxygen-like fashion, however, so that one day we can build up an easily accessible API reference. Kudos for getting the ball rolling with cregistry! I'd encourage you to do the same for registry2.0 and later on figure out what tool we use to produce the docs if doxygen can't handle the tcl format, what matters the most is that the documentation is already there in place. The new guide is also a great place to put this type of stuff in, but I'm figuring what we should really do is include this automagically generated API reference into the guide directly off the output of doxygen or a similar tool. So, in short, feel free to commit once you feel comfortable with your code for others to look at it, don't worry about anything else. Really looking forward to see this stuff go live!

Chris

Regards,... -jmpp

On Aug 3, 2007, at 4:19 PM, Chris Pickel wrote:

...

How would that script be run? Each user manually at their own discretion? Or through some upgrade glue internal to MacPorts? I would very much prefer the latter approach, something like my "upgrade" target in base/Makefile.in (later on I'll remove all that code once we consider enough time has gone by to upgrade most of our user base to the macports namespace, in the mean time I'm sure we can share that target for other upgrade purposes).

I had assumed an 'upgrade' target. However, since the script is external to the Makefile, it wouldn't be difficult to do the second method

When I said "internal to MacPorts" I was referring precisely to the upgrade target in base/Makefile.in (as opposed to a tool, some separate script, that the user has to run manually after reinstalling MacPorts, which is what I meant by the implied "external"). So I guess we're in "violent agreement" there ;-)

- i.e., at the end of mportinit, if file doesn't exist $ {registry.path}/registry/, source ${prefix}/share/macports/scripts/ reg_upgrade.tcl

I would personally prefer a Makefile target, MacPorts innards are already quite filled with upgrade hacks here and there to account for stuff we've changed in non backwards compatible ways throughout our history; but I'll let you be the judge for what'll work best for your upgrade. On a related note, cleaning up the legacy cruft would be lovely, IIRC registry1.0 is filled with it and I'm positive we don't need it any longer ;-)

OK. So to be clear, if I commit to trunk, there will be no trouble with keeping it out of 1.5.1? Then I'll go ahead and commit.

There shouldn't be, regardless of svnmerge.py if I have careful and detailed eyes. But now that you mention it, it's no secret that waiting until I merge to release_1_5 and release 1.5.1 will only make my life easier (I do have to analyze trunk to figure out what I'll merge). So if you're not pressed to commit then I guess waiting a couple of days will not hurt.

...

I'd encourage you to do the same for registry2.0 and later on figure out what tool we use to produce the docs if doxygen can't handle the tcl format, what matters the most is that the documentation is already there in place.

Alright. I think I'll probably use the doxygen style, but use /* instead of /**. Tools won't detect it as documentation, so they won't screw up when they try to associate it with the function signature ;)

Most cool! Having the documentation already in place will rock, later on we'll figure out how to extract it on an automated basis.

Chris

Thanks for this great work, Chris. Keep it up! Regards,... -jmpp