On Mon, 08/11/2010 18:32, Etienne URBAH wrote:
Balazs, Laurence, Sergio and all,
Inside the GLUE 2.0 information model, most entities are clearly persistent : Once an entity has been created, it exists until it is explicitly destroyed.
The exception is the 'Activity' entity, whose persistence is NOT clear. In particular :
A) Chapter 5.11 'Activity' states : For instance, a broker Service submits a Grid job to a selected execution Service; upon completion the execution Service submits a logging record to an accounting Service. Each of these Services may have associated an instance of a Grid Activity related to the lifecycle of the job within the service. All instances refer to the same conceptual job submitted by the user.
B) Chapter 6.9 'ComputingActivity' does NOT mention persistence, but the attributes of the 'ComputingActivity' entity have in fact various levels of lifetime and persistence. For example :
B.a) 'IDFromEndpoint', 'State', 'Error', 'Owner' and 'OtherMessages' have a meaning as soon as the ComputingActivity has been accepted by the endpoint, and should be kept in logging records with NO time limit.
B.b) 'WaitingPosition' has a meaning ONLY during the time when the ComputingActivity has been accepted by the ComputingManager. As soon as execution is finished, this attribute has NO meaning anymore.
B.c) 'LocalIDFromManager', 'LocalOwner' and 'ExecutionNode' have a direct meaning only during the time when the ComputingActivity is really executed by the ComputingManager. After the end of the execution, they only have an historical meaning, permitting to retrieve logging records inside the logs of the ComputingManager if needed.
B.d) 'UsedTotalWallTime', 'UsedTotalCPUTime' and 'UsedMainMemory' have a meaning as soon as the ComputingActivity has been accepted by the ComputingManager, and should be kept in accounting records (following UR GFD.98) with NO time limit.
B.e) 'ExitCode' and 'ComputingManagerExitCode' have a meaning only at the END of the execution of the ComputingActivity, and should be kept in accounting records (following the 'Activity Instance Documents Specification' proposal) with NO time limit.
C) Inside some implementations of grid middleware (notably gLite), the Information System has been designed to optimally manage a limited amount (a few MB) of information about the grid infrastructure, but NOT all logging and accounting records generated by Activities. So, the above mentioned attributes are NOT managed by the Information System, but by separate services, with adequate lifetime and persistence :
C.a) All attributes are managed directly by the ComputingService, but only during the execution of the ComputingActivity. As soon as execution is finished, the ComputingService forgets everything about the ComputingActivity.
C.b) All attributes which need persistence (as required for security audits) are saved on due time by the ComputingService inside logging and/or accounting records. Even after execution is finished, these logging and/or accounting records can be queried through dedicated logging and accounting services.
Finally, the current definition of the 'Activity' entity does NOT satisfactorily address the question of the lifetime and persistence of its attributes.
I imagine 2 solutions for addressing this question of lifetime and persistence (as required for security audits) of the attributes of the 'Activity' entity :
1) Keep the current 'Activity' entity unchanged ------------------------------------------------ The first solution is to keep the current 'Activity' entity unchanged, but clearly indicate that during its lifetime, any activity may be managed by various services, with various persistence. In particular, besides the service dedicated to its execution, each Activity should also managed by logging and accounting services for the persistence of its attributes.
This first solution keeps the current information model unchanged, but requires very precise coordination between the various services in order to simultaneously manage the SAME entity.
2) Split the current 'Activity' entity --------------------------------------- The second solution is to split the current 'Activity' entity into :
- A 'TransientActivity' entity managed during the adequate time span by the service dedicated to its execution,
- An 'ActivityRecord' entity whose instances are all attributes requiring persistence, with 2 attributes : - Type : Activity description (JSDL), Log record, Accounting record, ... - Value
This second solution breaks the current information model, but handles more clearly the lifetime of attributes, and is more flexible.
In both solutions, the 'Activity' entity is already managed at any time by at least one service. Then, is it really necessary that the Information Service ALSO has any knowledge of the ComputingActivity ? This is an open question.
Thank you in advance for : - studying this question of lifetime and persistence of the attributes of the 'Activity' entity, - providing explanations on points which I could misunderstand, - studying my 2 proposed solutions, - expressing your opinions.
Best regards.
----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------