Hello, I've been thinking about this proposal for the whole day; William has suggested me to write it here, to hear directly from you, to hear your feedbacks and, most thing important for me, your objections. Ah! I was about to forget, my name's Michelangelo De Simone, I'm very pleased to meet you all.:) Synopsis MacPortsDaemon (“MaPoD”) is an additional layer above the existing MacPorts utilities and API that provides asynchronous access to MacPorts functionalities (and new ones) to upper layer clients. It’s based on a pluggable architecture. Visionary scenarios Mike (limited user) wants to use an Adium version built on his own, to accomplish that he launches a MacPorts/MaPoD client, search for Adium in the search box, once he has found it he chooses to build and install it. The client confirms the action to Mike and giving him a notification that his version of Adium will be delivered directly into his ~/ Applications folder once the build process is over. The client becomes immediatly available for new commands. Amanda (limited user) already built Cyberduck but she needs it no more and then she decides to uninstall it. To accomplish that Amanda launches a MacPorts/MaPoD client, search for installed apps, once she has found Cyberduck she select it and mark it for removal. Her client confirms the action and becomes immediatly available for new commands. John (administrator on his own system) decides that Mike and Amanda need Gimp, so he decides to build and install it in order to let Amanda and Mike use it. To do that he launches a MacPorts/MaPoD client, search for Gimp; once he has found it he tells the client to install Gimp system wide, for users to use it. His client confirms the action and becomes immediatly available for new commands. Amanda (limited user) needs Gimp, so she starts looking for it with her MacPorts/MaPoD client and asks the client to build and install it. Client notifies Amanda that Gimp is already being built because of John choice to deploy it system wide and that she we’ll be able to use in some time; MacPorts/MaPoD will notify her with a Growl notification when Gimp will be ready to be used. Mike (limted user) asks himself how to remove objects and “stuff” produced by MacPorts/MaPoD and finds out that there’s no reason to be concerned about because MacPorts/MaPoD mantains itself scheduling maintance operations at the right time. Amanda (limted user) worries about tree sync and tries to manually update it using her client but it notifies her that the latest sync has been done automatically by MacPorts/MaPoD itself yesterday night. MaPoD Architecture Overview MaPoD aims to provide MacPorts additional features applying de- coupling among actual and future software components. See pic here: http://snipurl.com/22dyy The whole architecture of MaPoD shall be developed as a system service (LaunchDaemon), letting the clients at the upper layer request services asynchronously (“Build Adium, notify me when you are done”); users could also benefit from scheduled/automatic maintance operations such as self-update, application upgrades, tree syncs, objects cleanup. MaPoD Core ‘s task is to accomplish basic functionalities, wrapping those already provided by MacPorts’ port and adding them mechanism such scheduling, permission managment, event notifications (“Adium succesfully built by John”). No user on the system should sudo to build an app for his own use. MaPoD Interface is the real glue between MaPoD/MacPorts and GUI. Clients and MaPoD are completely loosly coupled because MaPod Interface provides a neutral way for them to communicate. Third party MacPorts developers could develop plugins to extend core functionalities (“Build Adium, notify me when you are done with a Growl notification” or “Build Adium, notify me when you are done with an email to jdoe@apple.com”). Conclusions With the appropriate choices any user on the same system, even limited ones, would see a full-featured MacPorts “distribution”; any user will have the possibility to search, install (from source or normal binary), remove his application with no touch to MacPorts at all. System administrator would also benefit from a fine grained permission system (“Do not let Mike build Adium” or “John can build and install only between 3pm and 9pm”). MacPorts installation would be “piloted” by MaPoD itself, so there will be no need to sudo-run it for occasional users. It’s clean, it’s “secure” (you know, Santa believes in security...:)). Whis this architecture possibilities are endless: how long would be needed to develop a plugin to mirror the tree among multiple nodes? How long to develop a web client and an adapter to manage the same build on multiple nodes? Obviuosly MacPorts core should never be used by users/clients, preferring MaPoD way to accomplish tasks. Back to the ground, there is much effort to spend and many issues to solve; first of all: how should MaPoD be launched, with whose privileges and permissions? Which technology should be used to develop all this stuff? Obj-C or C++ would be either good choices, once a good threading model has been decided (NSThread in 10.5 is a slightly forced choice). What about dependencies overlap? To respect decoupling how should MaPoD interface expose its own services? Stream/Socket interface? -- // Et quid amabo nisi quod rerum enigma est?
By the way, Michelangelo is proposing this in general, but is also looking to do part of this for GSoC. I wanted him to solicit feedback, especially since this idea touches on a lot of the tasks we're looking for in GSoC (privilege separation, GUIs, etc). -Bill On Mar 22, 2008, at 5:21 PM, Michelangelo wrote:
Hello,
I've been thinking about this proposal for the whole day; William has suggested me to write it here, to hear directly from you, to hear your feedbacks and, most thing important for me, your objections. Ah! I was about to forget, my name's Michelangelo De Simone, I'm very pleased to meet you all.:)
Synopsis MacPortsDaemon (“MaPoD”) is an additional layer above the existing MacPorts utilities and API that provides asynchronous access to MacPorts functionalities (and new ones) to upper layer clients. It’s based on a pluggable architecture.
Visionary scenarios Mike (limited user) wants to use an Adium version built on his own, to accomplish that he launches a MacPorts/MaPoD client, search for Adium in the search box, once he has found it he chooses to build and install it. The client confirms the action to Mike and giving him a notification that his version of Adium will be delivered directly into his ~/ Applications folder once the build process is over. The client becomes immediatly available for new commands.
Amanda (limited user) already built Cyberduck but she needs it no more and then she decides to uninstall it. To accomplish that Amanda launches a MacPorts/MaPoD client, search for installed apps, once she has found Cyberduck she select it and mark it for removal. Her client confirms the action and becomes immediatly available for new commands.
John (administrator on his own system) decides that Mike and Amanda need Gimp, so he decides to build and install it in order to let Amanda and Mike use it. To do that he launches a MacPorts/MaPoD client, search for Gimp; once he has found it he tells the client to install Gimp system wide, for users to use it. His client confirms the action and becomes immediatly available for new commands.
Amanda (limited user) needs Gimp, so she starts looking for it with her MacPorts/MaPoD client and asks the client to build and install it. Client notifies Amanda that Gimp is already being built because of John choice to deploy it system wide and that she we’ll be able to use in some time; MacPorts/MaPoD will notify her with a Growl notification when Gimp will be ready to be used.
Mike (limted user) asks himself how to remove objects and “stuff” produced by MacPorts/MaPoD and finds out that there’s no reason to be concerned about because MacPorts/MaPoD mantains itself scheduling maintance operations at the right time.
Amanda (limted user) worries about tree sync and tries to manually update it using her client but it notifies her that the latest sync has been done automatically by MacPorts/MaPoD itself yesterday night.
MaPoD Architecture Overview MaPoD aims to provide MacPorts additional features applying de- coupling among actual and future software components. See pic here: http://snipurl.com/22dyy
The whole architecture of MaPoD shall be developed as a system service (LaunchDaemon), letting the clients at the upper layer request services asynchronously (“Build Adium, notify me when you are done”); users could also benefit from scheduled/automatic maintance operations such as self-update, application upgrades, tree syncs, objects cleanup.
MaPoD Core ‘s task is to accomplish basic functionalities, wrapping those already provided by MacPorts’ port and adding them mechanism such scheduling, permission managment, event notifications (“Adium succesfully built by John”). No user on the system should sudo to build an app for his own use.
MaPoD Interface is the real glue between MaPoD/MacPorts and GUI. Clients and MaPoD are completely loosly coupled because MaPod Interface provides a neutral way for them to communicate.
Third party MacPorts developers could develop plugins to extend core functionalities (“Build Adium, notify me when you are done with a Growl notification” or “Build Adium, notify me when you are done with an email to jdoe@apple.com”).
Conclusions With the appropriate choices any user on the same system, even limited ones, would see a full-featured MacPorts “distribution”; any user will have the possibility to search, install (from source or normal binary), remove his application with no touch to MacPorts at all. System administrator would also benefit from a fine grained permission system (“Do not let Mike build Adium” or “John can build and install only between 3pm and 9pm”). MacPorts installation would be “piloted” by MaPoD itself, so there will be no need to sudo-run it for occasional users. It’s clean, it’s “secure” (you know, Santa believes in security...:)).
Whis this architecture possibilities are endless: how long would be needed to develop a plugin to mirror the tree among multiple nodes? How long to develop a web client and an adapter to manage the same build on multiple nodes?
Obviuosly MacPorts core should never be used by users/clients, preferring MaPoD way to accomplish tasks.
Back to the ground, there is much effort to spend and many issues to solve; first of all: how should MaPoD be launched, with whose privileges and permissions? Which technology should be used to develop all this stuff? Obj-C or C++ would be either good choices, once a good threading model has been decided (NSThread in 10.5 is a slightly forced choice).
What about dependencies overlap?
To respect decoupling how should MaPoD interface expose its own services? Stream/Socket interface? -- // Et quid amabo nisi quod rerum enigma est?
_______________________________________________ macports-dev mailing list macports-dev@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev
---- William Siegrist Software Support Engineer Mac OS Forge http://macosforge.org/ wsiegrist@apple.com 408 862 7337
On Mar 22, 2008, at 5:21 PM, Michelangelo wrote:
MacPortsDaemon (“MaPoD”) is an additional layer above the existing MacPorts utilities and API that provides asynchronous access to MacPorts functionalities (and new ones) to upper layer clients. It’s based on a pluggable architecture.
Hmmm. OK, so, I read through this description and I read through all the scenarios, then I went back and read the message from start to finish again, just to make sure I hadn't missed any of the subtle details, and my conclusion was the same as the first time: An additional layer to get what you want in the scenarios described sounds like over- engineering. All of the scenarios could just as easily be satisfied by another Cocoa port front-end that does the Growl notifications and the UI and everything else you've described. Such an application could also use the existing port(1) infrastructure directly, no intermediate layer or management daemon would be necessary to install / manage ports in the manner you describe (some of your per-user scenarios aren't particularly compelling, btw, seeing that macports pretty much assumes a single global $prefix and starts to break once you retarget ports piecemeal). If you wanted this capability on the command line, then things become a little different in design and scope, but you still don't need a separate daemon or layer to accomplish something similar to shell job control for multiple outstanding port operations. Don't get me wrong, I think something which lets you manage multiple in-flight operations and provides a good segregated status UI for managing them all is a fine idea, I simply think that a single application can accomplish those goals without needing to change or substantially augment the ports infrastructure at all. Given how limited a commodity time is, I also think that's a good thing. - Jordan
Il giorno 23/mar/08, alle ore 09:11, Jordan K. Hubbard ha scritto:
All of the scenarios could just as easily be satisfied by another Cocoa port front-end that does the Growl notifications and the UI and everything else you've described. Such an application could also use the existing port(1) infrastructure directly, no
Thanks for your answer Jordan, hearing your opinion is very important to me. You are right, a classical Cocoa app would solve some of the basic tasks I mentioned, if all that we want is having MacPorts being used by one (experienced) user:); but this would lead to some major challenge once someone will want to setup another client, perhaps a web-based one or a Java-based one (it's just an example:)): he shall write from scratch the entire lower layers of his app to make it work the way he wants, dealing with permissions as well (Task 7). No word about cuncurrent MacPorts running on the same system or the fact that a solution of this kind would be future-proof, in case MacPorts evolved. Generally speaking Task 5 ("GUI") would be easily addressed with an upper layer to manage MacPorts Core. One of the reasons for all this is to make MacPorts easy and affordable even to "naive users" with the help of third party developers who could count on an efficient and no trouble way to use the local MacPorts installation. As you can see I'm speaking quite generally without touching any techincal issue in detail, so I'd really be happy to hear more of your opinions, "just in case".:) Anyway thank you and don't be concerned, I really appreciate direct approaches.:) -- // Et quid amabo nisi quod rerum enigma est?
Michelangelo wrote:
Hello,
Hello Michelangelo,
I've been thinking about this proposal for the whole day; William has suggested me to write it here, to hear directly from you, to hear your feedbacks and, most thing important for me, your objections. Ah! I was about to forget, my name's Michelangelo De Simone, I'm very pleased to meet you all.:)
I read your proposal careful to make sure I fully understand it. But I added comments on items where I think this can be solved in other ways or are already possible.
Synopsis MacPortsDaemon (“MaPoD”) is an additional layer above the existing MacPorts utilities and API that provides asynchronous access to MacPorts functionalities (and new ones) to upper layer clients. It’s based on a pluggable architecture.
So this should be a daemon running in the background. I don't like the idea of running compile jobs in the background. Also, I don't see real advantages from it, see comments below.
Visionary scenarios Mike (limited user) wants to use an Adium version built on his own, to accomplish that he launches a MacPorts/MaPoD client, search for Adium in the search box, once he has found it he chooses to build and install it. The client confirms the action to Mike and giving him a notification that his version of Adium will be delivered directly into his ~/ Applications folder once the build process is over. The client becomes immediatly available for new commands.
So this is mostly a request for running MacPorts as non-root. This could really be improved, for example with an easier installation. Anders already made some additions to make it possible. But currently it needs ./configure flags which are not obvious.
Amanda (limited user) already built Cyberduck but she needs it no more and then she decides to uninstall it. To accomplish that Amanda launches a MacPorts/MaPoD client, search for installed apps, once she has found Cyberduck she select it and mark it for removal. Her client confirms the action and becomes immediatly available for new commands.
Comments below.
John (administrator on his own system) decides that Mike and Amanda need Gimp, so he decides to build and install it in order to let Amanda and Mike use it. To do that he launches a MacPorts/MaPoD client, search for Gimp; once he has found it he tells the client to install Gimp system wide, for users to use it. His client confirms the action and becomes immediatly available for new commands.
Already possible. If you add /opt/local/bin to the system wide PATH, software installed by MacPorts is available for all users.
Amanda (limited user) needs Gimp, so she starts looking for it with her MacPorts/MaPoD client and asks the client to build and install it. Client notifies Amanda that Gimp is already being built because of John choice to deploy it system wide and that she we’ll be able to use in some time; MacPorts/MaPoD will notify her with a Growl notification when Gimp will be ready to be used.
I don't think this will happen often. Most Mac OS X installations are used with a single user only and especially not with multiple users at a time. How many times will a user request software to be installed while a system-wide build is running? Also, why should a user not be allowed to compile his/her own version (maybe even newer)?
Mike (limted user) asks himself how to remove objects and “stuff” produced by MacPorts/MaPoD and finds out that there’s no reason to be concerned about because MacPorts/MaPoD mantains itself scheduling maintance operations at the right time.
Who wants jobs kicking in at random time and removing stuff? Also, there is no stuff left from a successful build. Users may decide to keep stuff from unsuccessfull builds in order to try to fix it.
Amanda (limted user) worries about tree sync and tries to manually update it using her client but it notifies her that the latest sync has been done automatically by MacPorts/MaPoD itself yesterday night.
Install a cronjob. No need for a daemon to accomplish this task.
MaPoD Architecture Overview MaPoD aims to provide MacPorts additional features applying de- coupling among actual and future software components. See pic here: http://snipurl.com/22dyy
Do you really think anyone wants a web interface for compiling stuff? I doubt that. I also doubt that anyone will write additional clients - for what purpose?
The whole architecture of MaPoD shall be developed as a system service (LaunchDaemon), letting the clients at the upper layer request services asynchronously (“Build Adium, notify me when you are done”); users could also benefit from scheduled/automatic maintance operations such as self-update, application upgrades, tree syncs, objects cleanup.
Leaving out the daemon thing here, you are requesting an API for these task. I think you are right; currently the API does not export these functionality.
MaPoD Core ‘s task is to accomplish basic functionalities, wrapping those already provided by MacPorts’ port and adding them mechanism such scheduling, permission managment, event notifications (“Adium succesfully built by John”). No user on the system should sudo to build an app for his own use.
As noted above, using MacPorts as non-root needs improvement, but can already be done.
MaPoD Interface is the real glue between MaPoD/MacPorts and GUI. Clients and MaPoD are completely loosly coupled because MaPod Interface provides a neutral way for them to communicate.
Third party MacPorts developers could develop plugins to extend core functionalities (“Build Adium, notify me when you are done with a Growl notification” or “Build Adium, notify me when you are done with an email to jdoe@apple.com”).
What kind of plugins do you think of? Adding a general hook to the API "run-when-ready" would be enough, I think.
Conclusions With the appropriate choices any user on the same system, even limited ones, would see a full-featured MacPorts “distribution”; any user will have the possibility to search, install (from source or normal binary), remove his application with no touch to MacPorts at all. System administrator would also benefit from a fine grained permission system (“Do not let Mike build Adium” or “John can build and install only between 3pm and 9pm”). MacPorts installation would be “piloted” by MaPoD itself, so there will be no need to sudo-run it for occasional users. It’s clean, it’s “secure” (you know, Santa believes in security...:)).
Add cronjobs, set permissions using chmod... No need to engineer a full new system? Why would anyone prohibit to install a specific port? Users can always build them outside MacPorts or install a binary. Also see the comment above about multi-user installations. As noted, non-root is possible.
Whis this architecture possibilities are endless: how long would be needed to develop a plugin to mirror the tree among multiple nodes? How long to develop a web client and an adapter to manage the same build on multiple nodes?
Mirror the tree? Run rsync and done? I don't get the point here. I already commented on a web interface above. If you want to build software for use on multiple nodes, compile it on one and create archives from it (already possible with MacPorts) and distribute it in your preferred way. Or provide archives to be installed on the clients.
Obviuosly MacPorts core should never be used by users/clients, preferring MaPoD way to accomplish tasks.
A user should never call the API manually also, what's the point? Users use clients like port on the command line or a GUI. I don't see any advantage from compiling in the background with a daemon. A GUI could provide queued builds which fits your request for "giving control back immediately." So, after all I don't see any real improvements for MacPorts from your proposal to add another layer. To achieve most parts of your proposal, install MacPorts system-wide and let the systems administrator compile software every user needs. Now every user itself can decide to also install MacPorts on his/her own in his/her home directory and install software there. No need to engineer new stuff. I don't think this needs to be synchronized, as I doubt anybody will install software at the same time as the system administrator does. Th whole point of installing software yourself is that you need another version or some other variants. And this synchronization is the only point which can't be achieved here. But is it really needed? My opinion is, what MacPorts would really need is a good API to be used by clients. The command line client should use the same API as any GUI client would use. This would need separation of interface and logic. And I see your request was merely a good GUI client which provides queuing, growl notifications and all the functionality MacPorts command line interface does. No need for a daemon to achieve this. Rainer
On Mar 23, 2008, at 11:34 AM, Michelangelo De Simone wrote:
Thanks for your answer Jordan, hearing your opinion is very important to me. You are right, a classical Cocoa app would solve some of the basic tasks I mentioned, if all that we want is having MacPorts being used by one (experienced) user:); but this would lead to some major challenge once someone will want to setup another client, perhaps a web-based one or a Java-based one (it's just an example:)): he shall write from scratch the entire lower layers of his app to make it work the way he wants, dealing with permissions as well (Task 7). No word about cuncurrent MacPorts running on the same system or the fact that a solution of this kind would be future- proof, in case MacPorts evolved. Generally speaking Task 5 ("GUI") would be easily addressed with an upper layer to manage MacPorts Core.
OK, so, let's take your points on order: 1. A Cocoa app would actually be aimed at the inexperienced user, not the experienced one, so I'm not sure I see your point. It's also an unquestionable fact that 99.9% of all Macs are single user (even on an XServe sitting in a rack somewhere, it's a single admin configuring it to provide services to multiple people) so there's no need to add per- user ACLs or have per-user interfaces. 2. Your raise the issue of multiple types of clients (Cocoa, web, etc) and that's a fair one, particularly if the ports collection on a given machine is going to need to be administered both locally and remotely, but that's more of an argument for making the existing set of port(1) APIs more robust and/or complete than adding an additional layer. In other words, if we really want to support multiple interfaces (and writing a web interface is a very different challenge, with different constraints, than a Cocoa UI) then I think the right approach isn't to create a complex interpositioning layer with a daemon managing the collection, the right approach is to provide the right port APIs for making the creation of such alternative UIs easier and more straight- forward. I also say this because the existing Tcl / C APIs for MacPorts were designed with this kind of thing very much in mind (the ui_foo APIs, for example, are implemented as abstractions for just this reason), though their evolution may not be (and probably is not) complete. The shortest distance between where we are and where you want to go would, therefore, seem to be to enhance what we have rather than create another layer. Again, MacPorts is already architected with embedding / external management in mind, so the question would be more one of "how can I make it do what I want" vs "how can I create another management layer." I see Rainer has already said many of the same things, so I'm clearly not alone. :-)
Anyway thank you and don't be concerned, I really appreciate direct approaches.:)
I'm glad to hear that since those are the kind I like best. :-) I also really don't want you to read all this as "they don't like my ideas!" or "clearly they do not see how another indirection layer could be useful", since I think we DO like your ideas and, with our academic researcher hats on, can always see the potential application for another abstraction layer (dozens of such layers can, in fact, be easily imagined if you put your mind to it). What we're trying to say is that it's also sometimes very important to focus purely on the goal and see how it can be done with the least amount of time and effort, then pick an approach which is somewhere in between "most flexible" and "quickest" if you want to successfully create software that people actually use. I think your first approach is too oriented towards flexibility and not enough towards the pragmatic and more quickly achievable. - Jordan
participants (5)
-
Jordan K. Hubbard
-
Michelangelo
-
Michelangelo De Simone
-
Rainer Müller
-
William Siegrist