All, (Note: I started this email before the call - I will finish it but do not believe it will change any minds.) I think defining IDL's will lead to non-interoperation without significant work on wire format .. think CORBA and its failure until IIOP happened. That said, here is a concrete proposal. Start with BES AS IS Create a new porttype BES_Extensions that defines two new functions ActivityResonse [] CreateActivitySet(ActivityDocument []) - based upon the BES create activity The activity response is an EPR or a SOAP 1.1 fault message element Define a StateTransferElement to be a <EPR, Destination_state) where destination state is a string formatted as per the BES specification Define a TransitionResponse to be a <new_state_string | SOAP1.1 fault message) TransitionResponse [] StateTransfer (StateTransferElement []) Profile ActivityEndpoint porttype MUST support getResourceProperties MUST support RNS:list RNS:list MUST return a predefined set of <name, EPR> pairs that contain the job information people want, e.g., session_directory, status Then create a PGI-Basic-Profile Conformance targets MUST handle HPC-BP, HPC-FSE (implies MUST support BES 1.0 FactoryPortType and JSDL) MUST support BES-Extensions MUST return EPRs to resources that support ActivityEndpoint MUST support state model profiles for staging, suspend, whatever_you_want_to_call_the_states_for_ARC if corresponding mechanism is supported MUST support ActivityDocument element <StartInPending> MUST/MAY support notification - we can argue this out - I do not have a strong feeling - we support it, but we also poll I would suggest that we have a separate conformance claim supporting RNS directly. This leaves one big question: How to handle GLUE? For this to be answered requires that the specific elements of GLUE2 to be indentified. Keep in mind the BES authors anticipated GLUE finalizing at some point. That said we either simply add the desired GLUE XML attributes to the results returned from getFactoryAttributesDocument, create a new function that returns just the GLUE, or perhaps better yet support getResourceProperties (I'd recommend using OGSA-WSRF-BP, it is supported in Teragrid and caGrid in the US, also I think by Unicore, and by Genesis II) and return the GLUE XML as resource properties. A
Hello, On Wednesday 20 May 2009 18:46, Andrew Grimshaw wrote:
All,
(Note: I started this email before the call - I will finish it but do not believe it will change any minds.)
I think defining IDL's will lead to non-interoperation without significant work on wire format .. think CORBA and its failure until IIOP happened.
That said, here is a concrete proposal.
Start with BES AS IS
Create a new porttype BES_Extensions that defines two new functions
ActivityResonse [] CreateActivitySet(ActivityDocument []) - based upon the BES create activity
The activity response is an EPR or a SOAP 1.1 fault message element
Define a StateTransferElement to be a <EPR, Destination_state) where destination state is a string formatted as per the BES specification
I suggest (as it was already suggested) to add optional Source_state for specifying state of activity which is expected to be when request is received. It's purpose is to avoid race condition.
Define a TransitionResponse to be a <new_state_string | SOAP1.1 fault message)
TransitionResponse [] StateTransfer (StateTransferElement [])
Profile ActivityEndpoint porttype
Did You mean BES-Activity porttype as defined in BES specs? If yes then it shoudl probablt be to create BES-Activity porttype. BES specs does not define any WSDL or schema for that port type. And few words in plain English is not enough to define one.
MUST support getResourceProperties
Do You mean OASIS WSRF-RP getResourceProperties? Just clarifying. Which properties are going to be provided?
MUST support RNS:list
RNS:list MUST return a predefined set of <name, EPR> pairs that contain the job information people want, e.g., session_directory, status
I would prefer to have that information profiled directly into EPR of activity. That would save a need to call at least one more operation for every initiated Activity.
Then create a PGI-Basic-Profile
Conformance targets
MUST handle HPC-BP, HPC-FSE (implies MUST support BES 1.0 FactoryPortType and JSDL)
MUST support BES-Extensions
Which?
MUST return EPRs to resources that support ActivityEndpoint
I assume ActivityEndpoint is BES-Activity. Otherwise ignore rest. I'm not sure that You mean by that. Do You suggest Address element of EPR point to URL where BES-Activity porttype is supported or are those EPRs to be sent to BES-Factory as defined in BES specs.
MUST support state model profiles for staging, suspend, whatever_you_want_to_call_the_states_for_ARC if corresponding mechanism is supported
How about client induced state transitions which are out of BES model?
MUST support ActivityDocument element <StartInPending>
In ARC we also support delayed start of activity processing. I guess other projects could come with more elements to be put there.
MUST/MAY support notification - we can argue this out - I do not have a strong feeling - we support it, but we also poll
I would suggest that we have a separate conformance claim supporting RNS directly.
This leaves one big question: How to handle GLUE?
For this to be answered requires that the specific elements of GLUE2 to be indentified.
Keep in mind the BES authors anticipated GLUE finalizing at some point.
That said we either simply add the desired GLUE XML attributes to the results returned from getFactoryAttributesDocument, create a new function that returns just the GLUE, or perhaps better yet support getResourceProperties (I'd recommend using OGSA-WSRF-BP, it is supported in Teragrid and caGrid in the US, also I think by Unicore, and by Genesis II) and return the GLUE XML as resource properties.
In ARC we are using OASIS WSRF-RP for providing Glue2 document as one of properties. We are using own set of properties but can probably switch to OGSA-WSRF-BP. A.K.
Good questions. Answers embedded.
-----Original Message-----
From: pgi-wg-bounces@ogf.org [mailto:pgi-wg-bounces@ogf.org] On Behalf Of
Aleksandr Konstantinov
Sent: Thursday, May 21, 2009 6:01 AM
To: pgi-wg@ogf.org
Subject: Re: [Pgi-wg] YAP - Yet Another Proposal
Hello,
On Wednesday 20 May 2009 18:46, Andrew Grimshaw wrote:
All,
(Note: I started this email before the call - I will finish it but do
not
believe it will change any minds.)
I think defining IDL's will lead to non-interoperation without
significant
work on wire format .. think CORBA and its failure until IIOP happened.
That said, here is a concrete proposal.
Start with BES AS IS
Create a new porttype BES_Extensions that defines two new functions
ActivityResonse [] CreateActivitySet(ActivityDocument []) - based upon
the
BES create activity
The activity response is an EPR or a SOAP 1.1 fault message element
Define a StateTransferElement to be a <EPR, Destination_state) where
destination state is a string formatted as per the BES specification
I suggest (as it was already suggested) to add optional Source_state
for specifying state of activity which is expected to be when request
is received. It's purpose is to avoid race condition.
[Andrew Grimshaw] OK - and then fault if the activity was not in that state?
Define a TransitionResponse to be a <new_state_string | SOAP1.1 fault
message)
TransitionResponse [] StateTransfer (StateTransferElement [])
Profile ActivityEndpoint porttype
Did You mean BES-Activity porttype as defined in BES specs?
If yes then it shoudl probablt be to create BES-Activity porttype. BES
specs does not define any WSDL
or schema for that port type. And few words in plain English is not enough
to define one.
[Andrew Grimshaw] I was not clear. The porttype in the spec has no methods. The idea is to either overload the one there, or most likely, create a new porttype. During the debate over BES that was added in by the faction that wanted to be able to directly communicate with the activities. I think to be absolutely clear it may need to be a new porttype.
MUST support getResourceProperties
Do You mean OASIS WSRF-RP getResourceProperties? Just clarifying.
Which properties are going to be provided?
[Andrew Grimshaw] This is a hot-potato issue. The real problem is that it would be very handy beyond BES/PGI to have a single porttype for getting metadata/attributes. WSRF-RP, and the restricted profile on it in the OGSA-WSRF-BP (it restricts the class of queries among other things.) The original BES working group could not come to consensus on using WSRF-RP in any form. That is why BES has getFactoryAttributesDocument as a sort of very restricted getProperties. I think now would be a good time to clean up past mistakes. On the other hand, we could just leave it the way it is and have getFactoryAttributesDocument return a schema extended with the XML Glue elements we desire.
MUST support RNS:list
RNS:list MUST return a predefined set of <name, EPR> pairs that contain
the
job information people want, e.g., session_directory, status
I would prefer to have that information profiled directly into EPR of
activity.
That would save a need to call at least one more operation for every
initiated Activity.
[Andrew Grimshaw] I'm not sure what you mean. Do you mean into the metadata portion of the EPR? I think this may be difficult. Status information for example changes? The contents of the session_directory may change. Or do you have something else in mind?
Then create a PGI-Basic-Profile
Conformance targets
MUST handle HPC-BP, HPC-FSE (implies MUST support BES 1.0
FactoryPortType
and JSDL)
MUST support BES-Extensions
Which?
[Andrew Grimshaw] I mean the proposed "BES-Extensions" porttype described earlier.
MUST return EPRs to resources that support ActivityEndpoint
I assume ActivityEndpoint is BES-Activity. Otherwise ignore rest.
I'm not sure that You mean by that. Do You suggest Address element of EPR
point
to URL where BES-Activity porttype is supported or are those EPRs to be
sent to BES-Factory
as defined in BES specs.
[Andrew Grimshaw] Sorry for being unclear. Let me try to clarify. Create activity returns an EPR. EPR's refer to some resource endpoint. I'd suggesting that the resource endpoint MUST support the BES-Activity porttype.
MUST support state model profiles for staging, suspend,
whatever_you_want_to_call_the_states_for_ARC if corresponding mechanism
is
supported
How about client induced state transitions which are out of BES model?
[Andrew Grimshaw] I think one of the faults that should be specified for the statetransfer method should be something like "Unknown source state" or "Unknown destination state".
MUST support ActivityDocument element <StartInPending>
In ARC we also support delayed start of activity processing. I guess
other projects could come with more elements to be put there.
MUST/MAY support notification - we can argue this out - I do not have a
strong feeling - we support it, but we also poll
I would suggest that we have a separate conformance claim supporting RNS
directly.
This leaves one big question: How to handle GLUE?
For this to be answered requires that the specific elements of GLUE2 to
be
indentified.
Keep in mind the BES authors anticipated GLUE finalizing at some point.
That said we either simply add the desired GLUE XML attributes to the
results returned from getFactoryAttributesDocument, create a new
function
that returns just the GLUE, or perhaps better yet support
getResourceProperties (I'd recommend using OGSA-WSRF-BP, it is supported
in
Teragrid and caGrid in the US, also I think by Unicore, and by Genesis
II)
and return the GLUE XML as resource properties.
In ARC we are using OASIS WSRF-RP for providing Glue2 document as one of
properties. We are using own set of properties but can probably switch to
OGSA-WSRF-BP.
[Andrew Grimshaw] If you are using OASIS WSRF-RP I believe that you are supporting the OGSA-WSRF-RP as I believe it is a subset.
A.K.
_______________________________________________
Pgi-wg mailing list
Pgi-wg@ogf.org
Andrew Grimshaw wrote: [...]
MUST support RNS:list
RNS:list MUST return a predefined set of <name, EPR> pairs that contain
the
job information people want, e.g., session_directory, status
I would prefer to have that information profiled directly into EPR of
activity.
That would save a need to call at least one more operation for every
initiated Activity.
[Andrew Grimshaw] I'm not sure what you mean. Do you mean into the metadata portion of the EPR? I think this may be difficult. Status information for example changes? The contents of the session_directory may change. Or do you have something else in mind?
The GES must support client-initiated transfer of the input data. This means that the user creates an activity in "pending" state. The GES returns an activity identifier, plus additional information (e.g., a URI) pointing to a location when the user can upload input data needed by the activity, using a supported protocol (scp/gridftp/whatever). At this point the user can start the activity. Thus, in the "strawman" we defined the CreateActivity operation to return a structure to avoid another call to the service to get the URI of the data staging area. This was considered as an implementation detail, which I certainly agree. On the other hand, would we have written something like "the GES must be efficient and avoid redundant interactions", that requirement would have been too generic and completely unverifiable, and again I would agree. I can not find any succint way of describing such a requirement, suggestions are welcome. Anyway, this specific problem would be easy to "fix" (by another profile), as the BES CreateActivity operation returns an ActivityDocument datatype, which contains an extension point. I also see that so far the discussion focused on adding features, but it would be equally important to get rid of stuff which is no longer needed. For example, the BES GetFactoryAttributesDocument as defined in BES makes little sense if we switch to a GLUE-based information model. From the discussion on wednesday I understand that it is not possible to deprecate operations; for GetFactoryAttributesDocument it is not possible to have it return an empty structure, because the normative BES specification says that the GetFactoryAttributesDocument operation returns a FactoryResourceAttributesDocument element, which must contain at least the following: - IsAcceptingNewActivities - TotalNumberOfActivities - TotalNumberOfContainedResources - NamingProfile - LocalResourceManagerType I did not check the other profiles/specifications, but I think that there could be other situations in which there are features not needed nor requested by GES which must be implemented anyway. So, while it would be possible to extend FactoryResourceAttributesDocument to include an XML rendering of some GLUE information, the same structure would still contain at least the abovementioned legacy stuff. Is there any way this can be avoided? Moreno. -- Moreno Marzolla INFN Sezione di Padova, via Marzolo 8, 35131 PADOVA, Italy EMail: moreno.marzolla@pd.infn.it Phone: +39 049 8277103 WWW : http://www.dsi.unive.it/~marzolla Fax : +39 049 8756233
Hi PGI team,
- From the discussion on wednesday I understand that it is not possible to deprecate operations;
------Original Message----- -From: pgi-wg-bounces@ogf.org [mailto:pgi-wg-bounces@ogf.org] On Behalf Of -Moreno Marzolla -Sent: Friday, May 22, 2009 12:47 PM -To: Andrew Grimshaw -Cc: 'Aleksandr Konstantinov'; pgi-wg@ogf.org -Subject: Re: [Pgi-wg] YAP - Yet Another Proposal - -Andrew Grimshaw wrote: -[...] ->>> MUST support RNS:list -> -> ->>> RNS:list MUST return a predefined set of <name, EPR> pairs that contain -> ->> the -> ->>> job information people want, e.g., session_directory, status -> -> -> ->> I would prefer to have that information profiled directly into EPR of -> ->> activity. -> ->> That would save a need to call at least one more operation for every -> ->> initiated Activity. -> -> [Andrew Grimshaw] I'm not sure what you mean. Do you mean into the
Well, I'm might be wrong here - but I understand from "profiling" that we can profile elements out, e.g. one operation (which might be very close to deprecate it). Hence, implementations that are PGI-compliant don't need to implement these old "profiled out" (deprecated) operations - but(!) as a consequence, are no longer BES-compliant at this point. However, the question arises of how much we have to profile out of BES to meet our requirements - and if there is something left in BES then: # New CreateActivity operation, because of immediate return of location et al. # get rid of management operations, because of non-needed overhead # new state model, it seems hard to meet requirements starting from the BES state model (work-in-progress) # keep activity portType, but add operations that are missing in the specification. It seems to me now there is fairly less left of BES itself, except the name, and even this one we wanted to change to Production Execution Service or Advanced Execution Service (back in Geneva). Again - a neutral observation. Take care, Morris ------------------------------------------------------------ Morris Riedel SW - Engineer Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Forschungszentrum Juelich Wilhelm-Johnen-Str. 1 D - 52425 Juelich Germany Email: m.riedel@fz-juelich.de Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel Phone: +49 2461 61 - 3651 Fax: +49 2461 61 - 6656 Skype: MorrisRiedel "We work to better ourselves, and the rest of humanity" Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe Vorstand: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender) metadata
-> portion of the EPR? I think this may be difficult. Status information for -> example changes? The contents of the session_directory may change. Or do you -> have something else in mind? - -The GES must support client-initiated transfer of the input data. This -means that the user creates an activity in "pending" state. The GES -returns an activity identifier, plus additional information (e.g., a -URI) pointing to a location when the user can upload input data needed -by the activity, using a supported protocol (scp/gridftp/whatever). At -this point the user can start the activity. - -Thus, in the "strawman" we defined the CreateActivity operation to -return a structure to avoid another call to the service to get the URI -of the data staging area. This was considered as an implementation -detail, which I certainly agree. On the other hand, would we have -written something like "the GES must be efficient and avoid redundant -interactions", that requirement would have been too generic and -completely unverifiable, and again I would agree. I can not find any -succint way of describing such a requirement, suggestions are welcome. - -Anyway, this specific problem would be easy to "fix" (by another -profile), as the BES CreateActivity operation returns an -ActivityDocument datatype, which contains an extension point. - -I also see that so far the discussion focused on adding features, but it -would be equally important to get rid of stuff which is no longer -needed. For example, the BES GetFactoryAttributesDocument as defined in -BES makes little sense if we switch to a GLUE-based information model. - From the discussion on wednesday I understand that it is not possible -to deprecate operations; for GetFactoryAttributesDocument it is not -possible to have it return an empty structure, because the normative BES -specification says that the GetFactoryAttributesDocument operation -returns a FactoryResourceAttributesDocument element, which must contain -at least the following: - -- IsAcceptingNewActivities -- TotalNumberOfActivities -- TotalNumberOfContainedResources -- NamingProfile -- LocalResourceManagerType - -I did not check the other profiles/specifications, but I think that -there could be other situations in which there are features not needed -nor requested by GES which must be implemented anyway. -So, while it would be possible to extend -FactoryResourceAttributesDocument to include an XML rendering of some -GLUE information, the same structure would still contain at least the -abovementioned legacy stuff. Is there any way this can be avoided? - -Moreno. - --- -Moreno Marzolla -INFN Sezione di Padova, via Marzolo 8, 35131 PADOVA, Italy -EMail: moreno.marzolla@pd.infn.it Phone: +39 049 8277103 -WWW : http://www.dsi.unive.it/~marzolla Fax : +39 049 8756233 - -_______________________________________________ -Pgi-wg mailing list -Pgi-wg@ogf.org -http://www.ogf.org/mailman/listinfo/pgi-wg
participants (4)
-
Aleksandr Konstantinov -
Andrew Grimshaw -
Moreno Marzolla -
Morris Riedel