I have seen all the recent mails that have been referenced in the correspondence. It could be a glitch in the mailer affecting just a part of the mailing list. It is difficult to say what happened. An action is probably not necessary, unless this is repeated, but I am cc-ing Stacy so she is aware of the glitch. -Hrabri -----Original Message----- From: owner-drmaa-wg@ggf.org [mailto:owner-drmaa-wg@ggf.org] On Behalf Of Andreas Haas Sent: Thursday, January 13, 2005 4:03 AM To: DRMAA Working Group Subject: Re: [drmaa-wg] C Binding After having asked Daniel, whom he replied with this mail I encounter mail from Roger wasn't distributed to drmaa-wg@ mailing list subscribers. Neither Daniel nor I actually received Roger's mail. Strange enough however Rogers mail shows up in DRMAA WG mailing list archieve under http://www-unix.gridforum.org/mail_archive/drmaa-wg/2005/01/msg00012.htm l has there been an outage with drmaa-wg@ mailing list distribution or is there a permanent problem? Andreas On Thu, 13 Jan 2005, Daniel Templeton wrote:

...

In a previous e-mail, Daniel Templeton wrote:

...

I've started to work on the C binding doc. There's something I've been dying to do since I started with DRMAA, and now that I have my hands on the C spec, I think it's a good time to do it.

Alarming attitude.

That's me. I'm a wild man. I also rip the tags off of matresses.

...

It is my understanding that the C spec is being revised with the intent of making it a stand-alone document. Although I would expect this to entail vast changes to the document, the technical content should be unchanged. This will make revision comparisons very difficult.

Please continue to propose changes during your editing cycle so they can be discussed and potentially incorporated into the spec after the changes to make it a stand-alone ducument.

A very reasonable suggestion. Sure.

...

...

That is namely to fix the drmaa_attr_*_t structures. They are currently unusable. In order to do anything useful with them, we need either a way to get the count of the elements in the structure or to reset the cursor to the beginning of the list or both.

Here's are the 6 new functions I propose:

int drmaa_get_num_attr_names(drmaa_attr_names_t* values, int* count); int drmaa_get_num_attr_values(drmaa_attr_values_t* values, int* count); int drmaa_get_num_attr_ids(drmaa_attr_ids_t* values, int* count); int drmaa_reset_attr_names(drmaa_attr_names_t* values); int drmaa_reset_attr_values(drmaa_attr_values_t* values); int drmaa_reset_attr_ids(drmaa_attr_ids_t* values);

I strongly recommend we at least add one set or the other. My preference would be both, but I think the first set is the more important.

The drmaa_get_next_* routines provide a mechanism to obtain every element in the data structure (once). I would expect many, if not most, callers of these routines to insert the elements into a data structure which is suitable for their usage. I am of the opinion that the existing routines are sufficient for incrementally constructed data structures. Although, if the caller is building something like an array, there is likely to be a resizing and/or memory copying overhead.

...

Does anyone have anything against adding these functions?

I don't think the convenience of the proposed functions is sufficient to require their implementation.

The agreed upon interface for the drmaa_get_next_* routines accept a buffer for the primary output value. For example: int drmaa_get_next_attr_name(drmaa_attr_names_t* values, char *value, size_t value_len); Thus, the drmaa implementation is not required to retain information once it has been provided to the application.

If it is important to iterate over values more than once, then presumably the DRMAA implementation should retain the entire set of values. I'd prefer to let the application decide when to retain values.

So, you're belief is that an implementation may only fetch values as required so it doesn't have to store them all in memory? I can see where that would be useful, but I don't see where that applies to DRMAA. The things you get back as one of these structs are the job ids from a drmaa_run_bulk_jobs() call, attribute names from drmaa_get_attribute_names(), and values from drmaa_get_vector_attribute_value(). In all of these cases, the implementation has to store the full list of values internally anyway, unless it writes everything into a database so that it can read it out incrementally, and even then it still at some point had to have the whole thing in memory so it could write it out in the first place (except maybe the job ids...). To me, this is a purely academic argument against a practical consideration.

I am in the peculiar position of both developing an implementation and using the implementation in applications. I have now written two "real" DRMAA apps, and both times I ran into a problem with not knowing how many elements were being returned to me. In the first app, I cheated and loaded the header that defined the structs so I could do the math myself. In the second app, I just declared a really big array, which is both a memory waste and dangerous. The correct answer would be to use a char** and re-malloc it if I get too many results, but that particular idiom is very error prone. I don't see why a developer should be forced to do dangerous and redundant things if it's avoidable. Besides, knowing how easy it would be to for the implementation to return the number of elements, it just seems sadistic to make developers jump through hoops for what should a simple operation. C is complicated enough without adding thorny interfaces.

To be fair, the number of ids returned from drmaa_run_bulk_jobs() can be calculated from start, end, and incr, but I see it as needlessly dangerous to make the developer do math to guess what the API is going to do, especially when that math is related to memory allocation. The less thinking the developer has to do, the fewer array overflows he's going to have.

This would all be different if we were talking about Java or C++ where there's a linked list implementation already there, just waiting to be used, but we're not. This is C, and when you're developing with sticks and stones, even the slightest gimmee can mean a great deal. Remember, adoption is important to us.

Daniel