Hi Hartmut, I certainly do understand your unease with implementation managed memory, and, yes, its the only place in the spec where we have that at the moment... The main (and actually only) reason for this is the ability to avoid memcopies. If you think of large messages, you definitely would like to have a way to avoid the copy from implementation memory (where it was received, and is buffered anyway) into application memory. For streams, that problem is not that bad, as you can delay the low level read until the stream.read() method is really called, and then you can read directly into the application level memory. For messages that does not work, as you do not know the size of the message beforehand. If you call test() (or check() now), the message must (at least partly) already recide in the application memory. Do you see any other possibility for avoiding the additional copy from impl. memory to application memory? About your second comment, the resuse of the msg (or message) object in other parts of the API: I am not sure about this, really. The main difference to stream and file byte buffers is the opacity of the message: for files and streams, you can write an int, five floats, another int etc. There are no contraints on when the data are transferred etc, and if, you flush the file. For the message API, the message is to be transferred as an entity, but can have internal structure. So you want to pack an int, five floats, and another int in a message, and then ship that as an opaque entity. w/o message object that is awkward. OTOH, a message object for that would be overkill for the file and stream API I think. However, I think we CAN reuse the message abstraction in several places, e.g. in the advert service, or, as we discussed in other mails, to create specific message types for certain application domains, or with typed data support etc. Well, these are my 2 cent... :-) Cheers, Andre. Quoting [Hartmut Kaiser] (Jan 20 2007):
Hi Andre,
here is the updated draft of the SAGA Message API, in preparation for OGF-19. It is also available in CVS, as usual.
We would be happy to get feedback on the list of course, so don't feel oblidged to hold back your comments for OGF-19 :-)
Sorry, it took me some time to digg through the new spec... I had to start implementing the stuff to have comments (I'm perhaps too old to do programming other than in an intuitive way :-P).
The comments I have relate to the API part of the spec.
Frankly, I don't like the design of the 'msg' object, I even think it's not viable to make it a first class SAGA object (derived from saga::object) - is there a use case requiring that?
The msg object in its current design mixes two different paradigms into one object (remember: every kind of bool in an API is a strong hint for a flawed design): a) saga::msg can be a wrapper object for application memory (size > 0) and b) saga::msg can be a object managing memory on behalf of the SAGA implementation (size == -1) I strongly believe that we don't need b) and even if we do, it should be implemented in a separate abstraction.
Why do I think we don't need b)?
Instead of relying on the SAGA implementation to manage the memory for the application (BTW: this is the only instance in the whole spec, where we require the implementation to do this), an application always may write (syntax is C++):
endpoint ep(...); ep.connect(...); int size = 0; if (ep.test(..., &size)) { void *data = new unsigned char[size]; ep.recv(data, size);
Or even better:
ep.recv(buffer(data, size)); ... delete data; }
i.e. The application always can handle the memory itself, without having to go through too much overhead.
This brings me to the second comment: by implementing the msg/buffer object solely for the purpose of wrapping up application memory this msg/buffer could be reused for other packages as well: file::read, file::write, stream::read, stream::write, rpc::parameter::buffer - to name a few.
BTW: I silently introduced a const_buffer and mutable_buffer objects, combined with a buffer() generator function in the C++ implementation of the saga::file package to improve memory handling in the API.
Regards Hartmut
-- "So much time, so little to do..." -- Garfield