Re: [Pgi-wg] OGF PGI Requirements - Flexibility and clarity versus Rigidity and confusion
Oxana, Well said.
I would add that I fear we may be trying too much to solve all problems
Aleksandr, Referring to your sentence/paragraph "Such "simple" job is very far from being "typical". At least in NorduGrid world AFAIK." Could you elaborate. I see in my work basically two "types" of jobs that dominate - sets of HTC "parameter space" jobs, and true parallel MPI jobs. In both cases the "job" is basically a command line - either an mpiexec/run, and application with parameters, or a script with parameters. The job has known inputs and outputs, or a directory tree location where it needs to run. The jobs runs to completion, or it fails, in either case there are output files and result codes. Sometimes the job is a workflow, but when you pick that apart it turns into jobs that have inputs and outputs along with a workflow engine orchestrating it all. What is a typical job that you see? When I say "typical" I mean covers 80% of the jobs. A -----Original Message----- From: Aleksandr Konstantinov [mailto:aleksandr.konstantinov@fys.uio.no] Sent: Sunday, May 02, 2010 3:36 PM To: pgi-wg@ogf.org Cc: Andrew Grimshaw; 'Oxana Smirnova'; 'Etienne URBAH'; 'David SNELLING'; lodygens@lal.in2p3.fr Subject: Re: [Pgi-wg] OGF PGI Requirements - Flexibility and clarity versus Rigidity and confusion Hello, I agree that problem is to difficult to solve. One should take into account that initially task was different. Originally AFAIR it was an attempt of few grid project to make a common interface suitable for them. Later those were forced to OGF and problem upgraded to almost unsolvable. Andrew Grimshaw at Saturday 01 May 2010 15:36 wrote: the
first time around - to "boil the ocean". To completely solve the whole problem is a daunting task indeed as there are so many issues.
I personally believe we will make more progress if we solve the minimum problem first, e.g., securely run a simple job from infrastructure/sw-stack A on infrastructure/sw-stack B.
This problem is already solved. And it was done in few ways. 1. Client stacks supporting multiple service stacks 2. BES + GSI 3. Other combinations currently in use And none is fully suitable for real production. So unless task of PGI is considered to be purely theoretical this approach would become equal to one more delay.
"Infrastructure/sw-stack A" means a set of resources (e.g., true parallel-Jugene, clusters, sets of desktops) running a middleware stack (e.g., Unicore 6 or Arc) configured a particular way. In the European context this might mean an NGI such as D-Grid with Unicore 6 running a job on NorduGrid running Arc. (Please forgive me if I have the particulars of the NGIs wrong.)
"Simple job" means a job that is typical, not special. This is not to say that its resource requirements are simple, it may have very particular requirements (cores per socket, interconnect, memory), rather I mean that the job processing required is simple: run w/o staging, simple staging,
Such "simple" job is very far from being "typical". At least in NorduGrid world AFAIK.
perhaps client interaction with the session directory pre, post, and during execution. Try to avoid complex job state models that will be hard to agree on, and difficult to implement in some environments.
"Securely" means sufficient authentication information required at B is provided to B in a form it will accept from a policy perspective. Further, that we try as much as possible to avoid a delegation definition that extends inwards beyond the outer boundary of a particular
I'm lost. Is is delegation or definition which extends?
infrastructure/sw-stack. (The last sentence is a bit awkward, I personally think that we will need to have two models of authentication and delegation - a legacy transport layer mechanism, and a message layer mechanism based on SAML, and that inside of a software stack we cannot expect sw-stacks to change their internal delegation mechanism.)
I believe authentication/delegation is the most critical item: if we cannot get the authentication/delegation issues solved, the rest is moot with respect to a PRODUCTION environment. We may be able to do demo's and stunts while punting on authentication/delegation, but we will not integrate production systems.)
Wasn't delegation voted no during last review? A.K.
Hi Andrew, all, allow me to start from very starters, to explain "typical" workload Aleksandr referred to. Both ARC and gLite "grew" from the requirements of the High Energy Physics community, more specifically - those of LHC experiments. I'll come back later to what it means in practice. The basic difference between gLite and ARC starting requirements is that gLite is designed for resources owned or controlled to a large extent by their users, while ARC is designed to support resources shared between different users and controlled by fairly independent resource providers. The immediate difference is policies: while gLite community is largely expected to comply with policies devised by the LHC Grid (WLCG) Joint Security and Policy Group, ARC community has no unique set of policies. ARC sites that contribute to LHC computing in general tend to respect the WLCG policies, but not too close, giving priority to local policies of resource owners. Needless to say, this introduces extra complexity into requirements, and reduces the rate of simple use cases. Now, LHC experiments are huge communities, between 500 and 3000 members in each. All well-educated computer-savvy people who never hesitate coming with their own brilliant solutions. Even within one experiment divergence is huge, and there are sites that support 4 experiments. This adds to the complexity: not only we have diverging or contradictory requirements from resource owners, we also have all sorts of diverging requirements from users. The least common denominator is 1: meaning "ping", because even "hello world" is ambiguous - where do you send the output, to a file or to a standard output? If it is standard output, do you write it to a log? Who said "hello world", the individual user or the whole experiment? Do we have a right to log individual activities at all? And so on. There are several basic kinds of typical jobs by LHC experiments, in general they can be separated in 5 categories: 1. Monte Carlo generation (no input, small resource consumption, small output) 2. Detector simulation (one input file, moderate resource consumption, moderate output) 3. Signal reconstruction (multiple input files, moderate resource consumption, large output) 4. Data merging (very many input files - like, 400, large resource consumption, large output) 5. Data analysis (huge amount of input files not necessarily known in advance, small resource consumption, small output) The job can have a number of states, e.g.: 1. Job is defined (may require authorisation) 2. Job matched to a site (requires authorisation, detailed site information, maybe data availability information) 3. Data are made available to a job (authorisation, probably delegation of data staging rights to a staging service) 4. Job processes data (CPU, I/O, network access to external databases requiring authorisation) 5. Job places data at [multiple] pre-defined destinations (authorisation, contacting external databases, probably delegation to a staging service) 6. Job is finished 7. Job has failed 8. Job is terminated (requires authorisation) 9. Job is resubmitted (authorisation, information) Each state may have a number of sub-states, depending on the experiment-specific framework. Authorisation may be per Virtual Organisation (file reading), per Role and/or Group within a VO (job types 1-4), per person (job type 5), or even per service (some frameworks accept services as VO members). Delegation of rights in general is very much needed, because of the large number of auxiliary services, distributed environment and general complexity of the workflow. No-one really knows how to achieve the goals without delegation. Each experiment has their own framework. Most such frameworks circumvent Grid services because they are too generic. This means that jobs are highly non-trivial as they attempt to re-implement Grid services such as information gathering and publishing, data movement and registration, resubmission etc.; they also tend to tweak authorisation by executing payload of users not authorised by Grid means. This complicates even further the job state picture. If PGI's outcome will make any of the above mentioned jobs impossible, most key ARC and gLite customers will not use PGI specs, and they will have only academic value. This was not the PGI goal, as "P" stands for "Production". Cheers, Oxana 13.05.2010 15:20, Andrew Grimshaw пишет:
Aleksandr, Referring to your sentence/paragraph
"Such "simple" job is very far from being "typical". At least in NorduGrid world AFAIK."
Could you elaborate. I see in my work basically two "types" of jobs that dominate - sets of HTC "parameter space" jobs, and true parallel MPI jobs. In both cases the "job" is basically a command line - either an mpiexec/run, and application with parameters, or a script with parameters. The job has known inputs and outputs, or a directory tree location where it needs to run. The jobs runs to completion, or it fails, in either case there are output files and result codes. Sometimes the job is a workflow, but when you pick that apart it turns into jobs that have inputs and outputs along with a workflow engine orchestrating it all.
What is a typical job that you see? When I say "typical" I mean covers 80% of the jobs.
A
-----Original Message----- From: Aleksandr Konstantinov [mailto:aleksandr.konstantinov@fys.uio.no] Sent: Sunday, May 02, 2010 3:36 PM To: pgi-wg@ogf.org Cc: Andrew Grimshaw; 'Oxana Smirnova'; 'Etienne URBAH'; 'David SNELLING'; lodygens@lal.in2p3.fr Subject: Re: [Pgi-wg] OGF PGI Requirements - Flexibility and clarity versus Rigidity and confusion
Hello,
I agree that problem is to difficult to solve. One should take into account that initially task was different. Originally AFAIR it was an attempt of few grid project to make a common interface suitable for them. Later those were forced to OGF and problem upgraded to almost unsolvable.
Oxana, Well said.
I would add that I fear we may be trying too much to solve all problems
Andrew Grimshaw at Saturday 01 May 2010 15:36 wrote: the
first time around - to "boil the ocean". To completely solve the whole problem is a daunting task indeed as there are so many issues.
I personally believe we will make more progress if we solve the minimum problem first, e.g., securely run a simple job from infrastructure/sw-stack A on infrastructure/sw-stack B.
This problem is already solved. And it was done in few ways. 1. Client stacks supporting multiple service stacks 2. BES + GSI 3. Other combinations currently in use And none is fully suitable for real production. So unless task of PGI is considered to be purely theoretical this approach would become equal to one more delay.
"Infrastructure/sw-stack A" means a set of resources (e.g., true parallel-Jugene, clusters, sets of desktops) running a middleware stack (e.g., Unicore 6 or Arc) configured a particular way. In the European context this might mean an NGI such as D-Grid with Unicore 6 running a job on NorduGrid running Arc. (Please forgive me if I have the particulars of the NGIs wrong.)
"Simple job" means a job that is typical, not special. This is not to say that its resource requirements are simple, it may have very particular requirements (cores per socket, interconnect, memory), rather I mean that the job processing required is simple: run w/o staging, simple staging,
Such "simple" job is very far from being "typical". At least in NorduGrid world AFAIK.
perhaps client interaction with the session directory pre, post, and during execution. Try to avoid complex job state models that will be hard to agree on, and difficult to implement in some environments.
"Securely" means sufficient authentication information required at B is provided to B in a form it will accept from a policy perspective. Further, that we try as much as possible to avoid a delegation definition that extends inwards beyond the outer boundary of a particular
I'm lost. Is is delegation or definition which extends?
infrastructure/sw-stack. (The last sentence is a bit awkward, I personally think that we will need to have two models of authentication and delegation - a legacy transport layer mechanism, and a message layer mechanism based on SAML, and that inside of a software stack we cannot expect sw-stacks to change their internal delegation mechanism.)
I believe authentication/delegation is the most critical item: if we cannot get the authentication/delegation issues solved, the rest is moot with respect to a PRODUCTION environment. We may be able to do demo's and stunts while punting on authentication/delegation, but we will not integrate production systems.)
Wasn't delegation voted no during last review?
A.K.
participants (2)
-
Andrew Grimshaw -
Oxana Smirnova