Re: [glue-wg] GLUE : Lifetime and persistence of the attributes of the 'Activity' entity
On Mon, 08/11/2010 18:32, Etienne URBAH wrote:
Balazs, Laurence, Sergio and all,
Inside the GLUE 2.0 information model, most entities are clearly persistent : Once an entity has been created, it exists until it is explicitly destroyed.
The exception is the 'Activity' entity, whose persistence is NOT clear. In particular :
A) Chapter 5.11 'Activity' states : For instance, a broker Service submits a Grid job to a selected execution Service; upon completion the execution Service submits a logging record to an accounting Service. Each of these Services may have associated an instance of a Grid Activity related to the lifecycle of the job within the service. All instances refer to the same conceptual job submitted by the user.
B) Chapter 6.9 'ComputingActivity' does NOT mention persistence, but the attributes of the 'ComputingActivity' entity have in fact various levels of lifetime and persistence. For example :
B.a) 'IDFromEndpoint', 'State', 'Error', 'Owner' and 'OtherMessages' have a meaning as soon as the ComputingActivity has been accepted by the endpoint, and should be kept in logging records with NO time limit.
B.b) 'WaitingPosition' has a meaning ONLY during the time when the ComputingActivity has been accepted by the ComputingManager. As soon as execution is finished, this attribute has NO meaning anymore.
B.c) 'LocalIDFromManager', 'LocalOwner' and 'ExecutionNode' have a direct meaning only during the time when the ComputingActivity is really executed by the ComputingManager. After the end of the execution, they only have an historical meaning, permitting to retrieve logging records inside the logs of the ComputingManager if needed.
B.d) 'UsedTotalWallTime', 'UsedTotalCPUTime' and 'UsedMainMemory' have a meaning as soon as the ComputingActivity has been accepted by the ComputingManager, and should be kept in accounting records (following UR GFD.98) with NO time limit.
B.e) 'ExitCode' and 'ComputingManagerExitCode' have a meaning only at the END of the execution of the ComputingActivity, and should be kept in accounting records (following the 'Activity Instance Documents Specification' proposal) with NO time limit.
C) Inside some implementations of grid middleware (notably gLite), the Information System has been designed to optimally manage a limited amount (a few MB) of information about the grid infrastructure, but NOT all logging and accounting records generated by Activities. So, the above mentioned attributes are NOT managed by the Information System, but by separate services, with adequate lifetime and persistence :
C.a) All attributes are managed directly by the ComputingService, but only during the execution of the ComputingActivity. As soon as execution is finished, the ComputingService forgets everything about the ComputingActivity.
C.b) All attributes which need persistence (as required for security audits) are saved on due time by the ComputingService inside logging and/or accounting records. Even after execution is finished, these logging and/or accounting records can be queried through dedicated logging and accounting services.
Finally, the current definition of the 'Activity' entity does NOT satisfactorily address the question of the lifetime and persistence of its attributes.
I imagine 2 solutions for addressing this question of lifetime and persistence (as required for security audits) of the attributes of the 'Activity' entity :
1) Keep the current 'Activity' entity unchanged ------------------------------------------------ The first solution is to keep the current 'Activity' entity unchanged, but clearly indicate that during its lifetime, any activity may be managed by various services, with various persistence. In particular, besides the service dedicated to its execution, each Activity should also managed by logging and accounting services for the persistence of its attributes.
This first solution keeps the current information model unchanged, but requires very precise coordination between the various services in order to simultaneously manage the SAME entity.
2) Split the current 'Activity' entity --------------------------------------- The second solution is to split the current 'Activity' entity into :
- A 'TransientActivity' entity managed during the adequate time span by the service dedicated to its execution,
- An 'ActivityRecord' entity whose instances are all attributes requiring persistence, with 2 attributes : - Type : Activity description (JSDL), Log record, Accounting record, ... - Value
This second solution breaks the current information model, but handles more clearly the lifetime of attributes, and is more flexible.
In both solutions, the 'Activity' entity is already managed at any time by at least one service. Then, is it really necessary that the Information Service ALSO has any knowledge of the ComputingActivity ? This is an open question.
Thank you in advance for : - studying this question of lifetime and persistence of the attributes of the 'Activity' entity, - providing explanations on points which I could misunderstand, - studying my 2 proposed solutions, - expressing your opinions.
Best regards.
----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------
Hi Etienne, The 'TransientActivity' model you propose in the second solution may go beyond the original intention of the information model as you will need to address the "lifespan" problem in the model itself, rather than letting the services choose what to do with the 'Activity' model. So to make it easier to implement and being more flexible in terms of what the services can do with the model, probably it is a good idea to keep it as it is right now. If we want to address the idea of persistence in general, it could be proposed for a future version of the model. Regards, David On Wed, May 16, 2012 at 3:09 PM, Etienne URBAH <urbah@lal.in2p3.fr> wrote:
On Mon, 08/11/2010 18:32, Etienne URBAH wrote:
Balazs, Laurence, Sergio and all,
Inside the GLUE 2.0 information model, most entities are clearly persistent : Once an entity has been created, it exists until it is explicitly destroyed.
The exception is the 'Activity' entity, whose persistence is NOT clear. In particular :
A) Chapter 5.11 'Activity' states : For instance, a broker Service submits a Grid job to a selected execution Service; upon completion the execution Service submits a logging record to an accounting Service. Each of these Services may have associated an instance of a Grid Activity related to the lifecycle of the job within the service. All instances refer to the same conceptual job submitted by the user.
B) Chapter 6.9 'ComputingActivity' does NOT mention persistence, but the attributes of the 'ComputingActivity' entity have in fact various levels of lifetime and persistence. For example :
B.a) 'IDFromEndpoint', 'State', 'Error', 'Owner' and 'OtherMessages' have a meaning as soon as the ComputingActivity has been accepted by the endpoint, and should be kept in logging records with NO time limit.
B.b) 'WaitingPosition' has a meaning ONLY during the time when the ComputingActivity has been accepted by the ComputingManager. As soon as execution is finished, this attribute has NO meaning anymore.
B.c) 'LocalIDFromManager', 'LocalOwner' and 'ExecutionNode' have a direct meaning only during the time when the ComputingActivity is really executed by the ComputingManager. After the end of the execution, they only have an historical meaning, permitting to retrieve logging records inside the logs of the ComputingManager if needed.
B.d) 'UsedTotalWallTime', 'UsedTotalCPUTime' and 'UsedMainMemory' have a meaning as soon as the ComputingActivity has been accepted by the ComputingManager, and should be kept in accounting records (following UR GFD.98) with NO time limit.
B.e) 'ExitCode' and 'ComputingManagerExitCode' have a meaning only at the END of the execution of the ComputingActivity, and should be kept in accounting records (following the 'Activity Instance Documents Specification' proposal) with NO time limit.
C) Inside some implementations of grid middleware (notably gLite), the Information System has been designed to optimally manage a limited amount (a few MB) of information about the grid infrastructure, but NOT all logging and accounting records generated by Activities. So, the above mentioned attributes are NOT managed by the Information System, but by separate services, with adequate lifetime and persistence :
C.a) All attributes are managed directly by the ComputingService, but only during the execution of the ComputingActivity. As soon as execution is finished, the ComputingService forgets everything about the ComputingActivity.
C.b) All attributes which need persistence (as required for security audits) are saved on due time by the ComputingService inside logging and/or accounting records. Even after execution is finished, these logging and/or accounting records can be queried through dedicated logging and accounting services.
Finally, the current definition of the 'Activity' entity does NOT satisfactorily address the question of the lifetime and persistence of its attributes.
I imagine 2 solutions for addressing this question of lifetime and persistence (as required for security audits) of the attributes of the 'Activity' entity :
1) Keep the current 'Activity' entity unchanged ------------------------------**------------------ The first solution is to keep the current 'Activity' entity unchanged, but clearly indicate that during its lifetime, any activity may be managed by various services, with various persistence. In particular, besides the service dedicated to its execution, each Activity should also managed by logging and accounting services for the persistence of its attributes.
This first solution keeps the current information model unchanged, but requires very precise coordination between the various services in order to simultaneously manage the SAME entity.
2) Split the current 'Activity' entity ------------------------------**--------- The second solution is to split the current 'Activity' entity into :
- A 'TransientActivity' entity managed during the adequate time span by the service dedicated to its execution,
- An 'ActivityRecord' entity whose instances are all attributes requiring persistence, with 2 attributes : - Type : Activity description (JSDL), Log record, Accounting record, ... - Value
This second solution breaks the current information model, but handles more clearly the lifetime of attributes, and is more flexible.
In both solutions, the 'Activity' entity is already managed at any time by at least one service. Then, is it really necessary that the Information Service ALSO has any knowledge of the ComputingActivity ? This is an open question.
Thank you in advance for : - studying this question of lifetime and persistence of the attributes of the 'Activity' entity, - providing explanations on points which I could misunderstand, - studying my 2 proposed solutions, - expressing your opinions.
Best regards.
------------------------------**----------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr ------------------------------**-----------------------
_______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
-- *David Horat* *Chief Information Officer* PLOCAN - Oceanic Platform of the Canary Islands +34 928 134 414 http://davidhorat.com/ La Información incluida en el presente correo electrónico es SECRETO PROFESIONAL Y CONFIDENCIAL, siendo para el uso exclusivo del destinatario arriba mencionado. Si usted lee este mensaje y no es el destinatario señalado, el empleado o el agente responsable de entregar el mensaje al destinatario, o ha recibido esta comunicación por error, le informamos que está totalmente prohibida cualquier divulgación, distribución o reproducción de esta comunicación, y le rogamos que nos lo notifique inmediatamente y nos devuelva el mensaje original a la dirección arriba mencionada. Gracias. The information contained in this e-mail is LEGALLY PRIVILEDGED AND CONFIDENTIAL and is intended only for the use of the addressee named above. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, or you have received this communication in error, please be aware that any dissemination, distribution or duplication of this communication is strictly prohibited, and please notify us immediately and return the original message to us at the address above. Thank you.
participants (2)
-
David Horat -
Etienne URBAH