Hi Alexander, thanks for your thoughts! In fact, that goes along the lines of several ongoing discussions - please see inserted comments below. Quoting [Alexander Beck-Ratzka] (Jun 21 2006):
Hello SAGA developers,
first of all: I am not taking part in developing SAGA, but I try to distribute GAT within the german D-Grid community project, and therefore I know about the SAGA efforts, and - as I feel - I also see some problems in the realisation...
Okay, let's start with the problems of GAT.
As GAT has started, it missed a lot of adaptors to the mostly ditributed grid middleware products. This is - at least in Germany - a fundamental reason for the very low acceptance of GAT in the Grid Community.
Understandable. What use is an API if it does not run on the environment in use...
As far as I know SAGA seems to inherit this deficiency of GAT. So Andre statet recently in a Email to me: "... we can definitively support only some adaptors, and we need the help of those projects, which have to develop AND SUPPORT adaptors to their middleware..."
That statement was in respect to the people who develop the C++ reference implementation. So, that group of people (currently Hartmut, Stephan and I) just don't have the resources secured to allow definite plans for any set of (well supoported) adaptors. OTOH, we are aware of that problem, and very much so, as we hade to deal with the same situaltion for GAT, as you mention. Now, there are two general ways to get middleware bindings:) (a) find funding to do it yourself, or (b) convince other groups to provide those. (a) Hartmut is currently funded from CCT, Stephan is writing his masters thesis, and has no secured funding after that right now. I will be funded for SAGA work from now on, but for a specific EU project. That obviously does not leave us much room for writing globus, EGEE, Unicore etc. bindings, and to support them OTOH, we try to get additional funding _dedicated_ to adaptor development: Shantenu tries to secure a small grant for that, and CCT tries to provide internal funding for some adaptors, to be used in their environment. It is doubtful if that will allow to cover support for more than one or two major middlewares. (b) How do you get other groups and projects to provide MW bindings? Two ways: (1) talk to their users: if their users demand SAGA for their middleware, they will provide and support bindings. (2) convince them that providing bindings is in their own interest: they find more users, and have an easier time supporting them. We try to address both: lobbying in application groups, and middleware provider groups. Currently, Ed and others are considering to found a 'SAGA Alliance' or something similar, to have a visible set of SAGA supporters, which might help to get critical mass. Also, we go to conferences and present SAGA, and plan to provide tutorials etc. to bring the API closer to application groups.
Please dont understand me wrong: I know about the small personel resources in the SAGA project, and that therefore only the development of a "SAGA Engine" and some local adaptors is possible. However, the missing of adaptors to Globus, Unicore or Condor etc.., is an urgent problem. The acceptance of software increases with its possibilities ot use it.
Absolutely agree. We are somewhat disappointed from DGrid in that respect: Yes, it is great that they consider to use GAT, but what we actually have been hoping for are some resources to develop GAT adaptors... If DGrid just looks at the API, and sees it useful, w/o acknowledging the need for support for the actual bindings, it won't fly. The same holds for other projects. Neither GAT nor SAGA are something which is 'ready to use' - right now its rather a framework, which you can use to get a SAGA API for your users in place. The story is different for Java CoG, hence their success...
If SAGA delivers now only an engine without a connection to any grid middleware, I would also not use SAGA to access e.g. Globus grdi functionality. Why should I develop an adaptor to my grid middleware (and even support it later on) for enabling the usage of SAGA; in such a case it's easier to develop directly against the middleware API.
No, that is wrong. Well, at least it is wrong if you consider the target community of SAGA: Grid Unaware Application developers! For you it might be the same, for some biology student, which tries to get his software running, it is not! Please don't confuse the groups we try to convince to provide SAGA bindings and implementations, with the group of people which is supposed to use it - in general, these groups are totally different...
If I develop an adaptor, I'll have to possible regions where problems can arise: SAGA and the access to the middleware API. Developing diretyl against the middleware will only bring problems with the middleware API. I believe that such preconditions will not lead to a wide acceptance of SAGA!
How to get rid of this problem? I like to suggest to deliver a SAGA toolkit. Beside the engine there also should be delivered a set of adaptors to the mostly distributed Grid middleware, as Globus, Unicore, Condor, etc... But how can this bed realised with the restricted resources? I have had the idea that the CCT should increase its support of the SAGA development, in Germany, there is no money available (at least right now). I don't know, if this is possible, but I'm quite shure that it must be a SAGA toolkit instead of a SAGA engine with local adaptors, if SAGA should be successful.
Hehe, nice - we would love to do that! Helas, you under estimate the amount of funding neccessary. The problem is actually not writing the adaptors, but to support and maintain them. In GAT, we had a number of adaptors written by students, which worked pretty well for a while. However, students leave, and after a while all you have left is a set of badly documented adaptors in mediocre quality, which break all over the place, and which eat your support resouces... We don't want to repeat that expereince (from GAT). So, you need more than a couple of students, and for more than a year! CCT does not have that kind of funding, nor has any institution alone. Also, we thing that the most proficient groups to write adaptors to some middleware are the developers of that middleware! For them, support of the adaptors is _much_ easier, and its also much simplier to maintain them over the development cycles of that middleware.
I hope, you understand my words as a helpful comment. I want SAGA to be successful, and therefore we have to think about a SAGA toolkit.
Alexander, thanks for your comments, and they truly point on the biggest acceptance problem we'll face: solid implementations. However, as said, we beleave that the ultimate goal should be to have the middleware developers supporting SAGA bindings to their API, and move that burden away from the application groups and projects. Cheers, Andre.
Cheers
Alexander
-- "So much time, so little to do..." -- Garfield