FW: ISN spec

antony.wilson＠stfc.ac.uk

15 Mar 2011 15 Mar '11

4:50 p.m.

Hi Andre Thanks for your feedback. The document has been updated http://hepunx.rl.ac.uk/egee/sa3-uk/sd/ISNSpec.pdf Regards Antony

...

- Use cases: the document is not very explicit about the requirements, and only vaguely refers to the SAGA use cases - I think the requirements should be defined in somewhat more detail, and/or one or more use cases should explicitly be mentioned.

Now mentions UC 1, section 7

...

- "No use-case has been identified for the operators >=, >, <=, < to be applied to strings. An Implementation wishing to support these comparison operators on strings MUST select a collation sequence. Alternatively, an implementation CAN treat all string comparisons as true, or reject them as invalid SQL."

IMHO, if the operators are defined, they should be evaluated, or an error should be thrown. I think it will 'surprise' users if the ops are accepted, but not applied.

I am inclined to agree but this is consistent with how the SD spec describes the treatment of strings

...

- the classes do not inherit the saga::async interface, and thus will not have asynchronous operations. Is that on purpose, or an oversight?

Again this is to remain consistent with the SD spec

...

- "model: the name of the information model"

That seems to be a freeform string? I think it would be useful to at least propose (SHOULD) string constants for glue 1 and 2?

A note added

...

- "in session"

by convention in the core spec, the session parameter should always be the first one. It is still optional though - see core spec for details (basically, it is up to he language binding how to render that).

Updated

...

- You do not have TimeOut, PermissionDenied, Authorization/Authentication etc exceptions. I am unclear about the underlying protocol (and the API should not care), but are you sure those error modes do not (ever) apply to the calls?

- some calls have no exceptions whatsoever - are you sure they will *always* succeed? :-)

Added to constructor get_data and list_related_entity_names the data has already been loaded and parsed as a result of the constructor so will always succeed exceptions Timeout to get_related_entities these calls should return objects that you are authorized to see

...

- what is "N.B" in the notes for "list_related_entity_names" ?

Removed

...

- indentation seems sometimes faulty, in particular on line breaks in the verbatim sections, but also in the table in 3.1

Fixed

...

- in the example, the include should just be - at least we don't define individual includes for any other package so far.

Changed

...

- you list two prototypes of 'get_related_entities' in the IDL section, but only one set of details later on. I guess you could use a default parameter value for the filter in IDL to fix this?

Added comment saying that the operation is overloaded, this is then consistent with the SD spec

...

Hope that helps,

Andre.

On Sat, Feb 26, 2011 at 1:03 PM, Andre Merzky wrote:

...
Dear SAGA group,

on behalf of Steve Fisher and Antony Wilson, please consider the attached document. If there are no objections raised on this list within the next 10 days, we intent to submit this document to the OGF editor for publication as P-REC.

Thanks to Steve and Antony for working on this document! :-)

Best, Andre.

---------- Forwarded message ---------- From: Steve Fisher Date: Fri, Feb 25, 2011 at 11:36 PM Subject: ISN spec To: Shantenu Jha , Andre Merzky , Thilo Kielmann Cc: Antony Wilson

Hi,

Antony made the few small changes as requested at the last OGF and we would now like to circulate it for a final call within the group.

If you agree please do so - then after a suitable period it can go to the OGF editor.

It is at http://hepunx.rl.ac.uk/egee/sa3-uk/sd/ISNSpec.pdf

Steve

-- So much time, so little to do... [Garfield]

-- So much time, so little to do... [Garfield] -- saga-rg mailing list saga-rg@ogf.org http://www.ogf.org/mailman/listinfo/saga-rg

-- Scanned by iCritical.

Show replies by date

Andre Merzky

15 Mar 15 Mar

10:22 p.m.

Hi Antony, thanks to you, too, for the detailed replies! :-) Some comments below.... On Tue, Mar 15, 2011 at 5:50 PM, wrote:

...

...
- "No use-case has been identified for the operators >=, >, <=, < to be applied to strings. An Implementation wishing to support these comparison operators on strings MUST select a collation sequence. Alternatively, an implementation CAN treat all string comparisons as true, or reject them as invalid SQL."

IMHO, if the operators are defined, they should be evaluated, or an error should be thrown. I think it will 'surprise' users if the ops are accepted, but not applied.

I am inclined to agree but this is consistent with how the SD spec describes the treatment of strings

That makes sense, although one could argue that this points to a problem/oversight in the SD spec. But I have to admit that there are some places in the SAGA core API were we don't cleanly handle similar cases either. So, I guess we have, at least for the time being, to live with that. Anyway, you may want to reconsider if fixing this would really be a problem. The mentioned cases in the core spec are planned to get fixed at some point (FWIW).

...

...
- the classes do not inherit the saga::async interface, and thus will not have asynchronous operations. Is that on purpose, or an oversight?

Again this is to remain consistent with the SD spec

Again, that point makes certainly sense, but may again point to an oversight in the SD spec. What is the rationale behind that decision? All other packages support remote operations, as it basically does not carry any ballast on implementation side (once you have implemented a way to do remote ops, its relatively easy to apply that to all calls), and async operations seem very useful in distributed applications?

...

...
- You do not have TimeOut, PermissionDenied, Authorization/Authentication etc exceptions. I am unclear about the underlying protocol (and the API should not care), but are you sure those error modes do not (ever) apply to the calls?

- some calls have no exceptions whatsoever - are you sure they will *always* succeed? :-)

Added to constructor

get_data and list_related_entity_names the data has already been loaded and parsed as a result of the constructor so will always succeed

Well, you assume that your implementation fetches all data on the c'tor, and does no extra queries later on. But is that semantically prescribed by the API? I can't see that in the current spec - I think a stateless client could be implemented according to that API spec. IFF you prescribe such a cache (and thus client state), what is it's scope, and its lifetime? Infinite may be a valid lifetime value I guess, but I doubt one would cache related entities of related entities? While checking again: get_related_entities(): it is not well specified when BadParameter is thrown. The filter being ill-formed is obvious - but what happens on an related_name parameter which does not exist? Would a DoesNotExist exception possibly be a better match?

...

exceptions Timeout to get_related_entities these calls should return objects that you are authorized to see

Are no race conditions possible between list_related_entity_names and get_related_entities (name)? Like, could an entry name returned by the first call disappear in the backend database before the second call gets called?

...

...
- you list two prototypes of 'get_related_entities' in the IDL section, but only one set of details later on. I guess you could use a default parameter value for the filter in IDL to fix this?

Added comment saying that the operation is overloaded, this is then consistent with the SD spec

Yes, this makes sense. Best, Andre. -- So much time, so little to do... [Garfield]

antony.wilson＠stfc.ac.uk

18 Mar 18 Mar

3:02 p.m.

Hi Andre More changes made

...

Hi Antony,

thanks to you, too, for the detailed replies! :-) Some comments below....

...
...
- the classes do not inherit the saga::async interface, and thus will not have asynchronous operations. Is that on purpose, or an oversight?

Again this is to remain consistent with the SD spec

Again, that point makes certainly sense, but may again point to an oversight in the SD spec. What is the rationale behind that decision? All other packages support remote operations, as it basically does not carry any ballast on implementation side (once you have implemented a way to do remote ops, its relatively easy to apply that to all calls), and async operations seem very useful in distributed applications?

I believe a the time it was considered that call to sd would be relatively fast. Do you have a use case for making use of sd in an asynchronous manner?

...

...
...
- You do not have TimeOut, PermissionDenied, Authorization/Authentication etc exceptions. I am unclear about the underlying protocol (and the API should not care), but are you sure those error modes do not (ever) apply to the calls?

- some calls have no exceptions whatsoever - are you sure they will *always* succeed? :-)

Added to constructor

get_data and list_related_entity_names the data has already been loaded and parsed as a result of the constructor so will always succeed

Well, you assume that your implementation fetches all data on the c'tor, and does no extra queries later on. But is that semantically prescribed by the API? I can't see that in the current spec - I think a stateless client could be implemented according to that API spec.

IFF you prescribe such a cache (and thus client state), what is it's scope, and its lifetime? Infinite may be a valid lifetime value I guess, but I doubt one would cache related entities of related entities?

By the nature of our implementation these make no sense but I accept that They may be relevant to other implementations get_data() - added NoSucsess and Timeout exceptions list_related_entity_names() - added NoSucsess and Timeout exceptions

...

While checking again: get_related_entities(): it is not well specified when BadParameter is thrown. The filter being ill-formed is obvious - but what happens on an related_name parameter which does not exist? Would a DoesNotExist exception possibly be a better match?

get_related_entities(related_name, filter) -added DoesNotExist exception and a note

...

...
exceptions Timeout to get_related_entities these calls should return objects that you are authorized to see

Are no race conditions possible between list_related_entity_names and get_related_entities (name)? Like, could an entry name returned by the first call disappear in the backend database before the second call gets called?

No, the entity names returned from list_related_entity_names are those defined in the glue schema, if they change then we need a new model

...

...
Yes, this makes sense.

Best, Andre.

-- Scanned by iCritical.

Andre Merzky

4:03 p.m.

On Fri, Mar 18, 2011 at 4:02 PM, wrote:

...

Hi Andre

More changes made

Thanks, appreciated!

...

...
...
...
- the classes do not inherit the saga::async interface, and thus will not have asynchronous operations. Is that on purpose, or an oversight?

Again this is to remain consistent with the SD spec

Again, that point makes certainly sense, but may again point to an oversight in the SD spec. What is the rationale behind that decision? All other packages support remote operations, as it basically does not carry any ballast on implementation side (once you have implemented a way to do remote ops, its relatively easy to apply that to all calls), and async operations seem very useful in distributed applications?

I believe a the time it was considered that call to sd would be relatively fast. Do you have a use case for making use of sd in an asynchronous manner?

Latency issues are a general concern in distributed applications, and simply cannot be avoided. Just as an example: a project we work with does distributed runs over TeraGrid and DEISA. No matter how the app is configured, it needs to be able to cater for the cross atlantic latencies. Now, I agree that most use cases of SD do not cover high-frequent queries - but IIUC, the ISN explicitely caters for browsing an arbitrary information service interactively - that, IMHO, immediately implies latency concerns. Let me ask differently: what is the reason to *not* allow async ops? In general, that does not change the service interface, but allows the SAGA implementation to utilize threads for latency hiding. What is the downside?

...

...
...
...
- You do not have TimeOut, PermissionDenied, Authorization/Authentication etc exceptions. I am unclear about the underlying protocol (and the API should not care), but are you sure those error modes do not (ever) apply to the calls?

- some calls have no exceptions whatsoever - are you sure they will *always* succeed? :-)

Added to constructor

get_data and list_related_entity_names the data has already been loaded and parsed as a result of the constructor so will always succeed

Well, you assume that your implementation fetches all data on the c'tor, and does no extra queries later on. But is that semantically prescribed by the API? I can't see that in the current spec - I think a stateless client could be implemented according to that API spec.

IFF you prescribe such a cache (and thus client state), what is it's scope, and its lifetime? Infinite may be a valid lifetime value I guess, but I doubt one would cache related entities of related entities?

By the nature of our implementation these make no sense but I accept that They may be relevant to other implementations

get_data() - added NoSucsess and Timeout exceptions list_related_entity_names() - added NoSucsess and Timeout exceptions

Yes, that is the point - I understand that your implementation may never throw those. Thanks.

...

...
...
exceptions Timeout to get_related_entities these calls should return objects that you are authorized to see

Are no race conditions possible between list_related_entity_names and get_related_entities (name)? Like, could an entry name returned by the first call disappear in the backend database before the second call gets called?

No, the entity names returned from list_related_entity_names are those defined in the glue schema, if they change then we need a new model

Ah, so an entry name being returned does not imply that data for that entry exist in the backend? I.e., if there is an optional element in the glue schema, and the database does not have a value for that element, that element's name is still returned in the list? That was not clear to me, thanks. Best, Andre. -- So much time, so little to do... [Garfield]

Steve Fisher

10:31 p.m.

New subject: FW: ISN spec

On 18 March 2011 16:03, Andre Merzky wrote:

...

On Fri, Mar 18, 2011 at 4:02 PM, wrote:

...
Hi Andre

More changes made

Thanks, appreciated!

...
...
...
...
- the classes do not inherit the saga::async interface, and thus will not have asynchronous operations. Is that on purpose, or an oversight?

Again this is to remain consistent with the SD spec

Again, that point makes certainly sense, but may again point to an oversight in the SD spec. What is the rationale behind that decision? All other packages support remote operations, as it basically does not carry any ballast on implementation side (once you have implemented a way to do remote ops, its relatively easy to apply that to all calls), and async operations seem very useful in distributed applications?

I believe a the time it was considered that call to sd would be relatively fast. Do you have a use case for making use of sd in an asynchronous manner?

Latency issues are a general concern in distributed applications, and simply cannot be avoided. Just as an example: a project we work with does distributed runs over TeraGrid and DEISA. No matter how the app is configured, it needs to be able to cater for the cross atlantic latencies.

Now, I agree that most use cases of SD do not cover high-frequent queries - but IIUC, the ISN explicitely caters for browsing an arbitrary information service interactively - that, IMHO, immediately implies latency concerns.

Let me ask differently: what is the reason to *not* allow async ops? In general, that does not change the service interface, but allows the SAGA implementation to utilize threads for latency hiding. What is the downside?

It is only because we saw no benefit in adding the async complexity. If we add it to ISN then it really needs adding to SD as well. I will discuss the consequences directly with you in Taipei.

...

...
...
...
...
- You do not have TimeOut, PermissionDenied, Authorization/Authentication etc exceptions. I am unclear about the underlying protocol (and the API should not care), but are you sure those error modes do not (ever) apply to the calls?

- some calls have no exceptions whatsoever - are you sure they will *always* succeed? :-)

Added to constructor

get_data and list_related_entity_names the data has already been loaded and parsed as a result of the constructor so will always succeed

Well, you assume that your implementation fetches all data on the c'tor, and does no extra queries later on. But is that semantically prescribed by the API? I can't see that in the current spec - I think a stateless client could be implemented according to that API spec.

IFF you prescribe such a cache (and thus client state), what is it's scope, and its lifetime? Infinite may be a valid lifetime value I guess, but I doubt one would cache related entities of related entities?

By the nature of our implementation these make no sense but I accept that They may be relevant to other implementations

get_data() - added NoSucsess and Timeout exceptions list_related_entity_names() - added NoSucsess and Timeout exceptions

Yes, that is the point - I understand that your implementation may never throw those. Thanks.

...
...
...
exceptions Timeout to get_related_entities these calls should return objects that you are authorized to see

Are no race conditions possible between list_related_entity_names and get_related_entities (name)? Like, could an entry name returned by the first call disappear in the backend database before the second call gets called?

No, the entity names returned from list_related_entity_names are those defined in the glue schema, if they change then we need a new model

Ah, so an entry name being returned does not imply that data for that entry exist in the backend? I.e., if there is an optional element in the glue schema, and the database does not have a value for that element, that element's name is still returned in the list? That was not clear to me, thanks.

It would be very inefficient to look at the data to determine the set of related entities. This is just information from the schema. Steve

...

Best, Andre.

-- So much time, so little to do... [Garfield] -- saga-rg mailing list saga-rg@ogf.org http://www.ogf.org/mailman/listinfo/saga-rg

antony.wilson＠stfc.ac.uk

30 Mar 30 Mar

9:05 a.m.

Added async Added note the list_releted_entity_names is a schema operation Regards Antony -- Scanned by iCritical.

Steve Fisher

31 Mar 31 Mar

12:42 p.m.

New subject: FW: ISN spec

Antony, Could you also circulate a pointer to the document Steve On 30 March 2011 10:05, wrote:

...

Added async

Added note the list_releted_entity_names is a schema operation

Regards Antony

-- Scanned by iCritical. -- saga-rg mailing list saga-rg@ogf.org http://www.ogf.org/mailman/listinfo/saga-rg

antony.wilson＠stfc.ac.uk

1:23 p.m.

New subject: FW: ISN spec

The latest draft of the ISN spec can be found here: http://hepunx.rl.ac.uk/egee/sa3-uk/sd/ISNSpec.pdf

...

-----Original Message----- From: Steve Fisher [mailto:dr.s.m.fisher@gmail.com] Sent: 31 March 2011 13:42 To: Wilson, Antony (STFC,RAL,PPD) Cc: andre@merzky.net; saga-rg@ogf.org Subject: Re: [SAGA-RG] FW: ISN spec

Antony,

Could you also circulate a pointer to the document

Steve

On 30 March 2011 10:05, wrote:

...
Added async

Added note the list_releted_entity_names is a schema operation

Regards Antony

-- Scanned by iCritical. -- saga-rg mailing list saga-rg@ogf.org http://www.ogf.org/mailman/listinfo/saga-rg

-- Scanned by iCritical.

Andre Merzky

8:15 p.m.

Great, thanks! The document seems to be good to go to the editor, IMHO. Best, Andre. On Wed, Mar 30, 2011 at 11:05 AM, wrote:

...

Added async

Added note the list_releted_entity_names is a schema operation

Regards Antony

-- So much time, so little to do... [Garfield]

4937

Age (days ago)

4953

Last active (days ago)

List overview

Download

8 comments

3 participants

participants (3)

Andre Merzky
antony.wilson＠stfc.ac.uk
Steve Fisher

FW: ISN spec

antony.wilson＠stfc.ac.uk

Andre Merzky

antony.wilson＠stfc.ac.uk

Andre Merzky

Steve Fisher

antony.wilson＠stfc.ac.uk

Steve Fisher

antony.wilson＠stfc.ac.uk

Andre Merzky

tags

participants (3)