- ur-wg - lists.ogf.org

Out of Scope: Comment #5k
by Laura F McGinnis 24 May '06

24 May '06

2 2

Extensions: Comment #10
by Laura F McGinnis 24 May '06

24 May '06

2 2

Mailing List Discussions: Comment #14a
by Laura F McGinnis 24 May '06

24 May '06

2 2

Mailing List Discussions: Comment #3c
by Laura F McGinnis 18 May '06

18 May '06

4 5

Next few conference calls
by Stephen M Pickles 16 May '06

16 May '06

We agreed at GGF17 to hold conference calls every two weeks, at the same time. Accordingly I have booked the next three conference calls: Thursday 25 May 2006, 2pm BST (9am Pittsburgh) Thursday 2 June 2006, 2pm BST Thursday 16 June 2006, 2pm BST BST is British Summer Time (UTC+1). I have booked 6 lines for 55 minutes. The details are the same for each call: Primary Dial-in Number: 0844.888.8888 (UK 5p National Rate) Alternate Dial-in Numbers: +44.870.088.5706 (For Callers Outside of the U.K.) 01805.004.102 (For Callers in Germany) The Participant Access Code is: 61652. Best wishes, Stephen

3 3

Re: [ur-wg] Meeting Notes
by Matthew Ford 11 May '06

11 May '06

Quoting "Donal K. Fellows" <donal.k.fellows(a)manchester.ac.uk>: > Future version to extend to more complex things than batch system job > accounting. But I think think we have this now! Maybe I'm reading too much into the term "extend". It's much more a change of method and clarification that I would like to see. As you move through-out the document it becomes increasingly generalised. By section ten, I believe the current specs clear intention is to be able to account much more than a batch job. However it's not with out problem. Some thoughts and quotes: - Job Level Accounting from the Original Spec is supposed to work at the batch job level. It is restricted form of the usage of the base properties of section 3 or later the common properties of section 10. From Sec.4: "Job level accounting reports accounting data at the job level. PBS and LoadLeveller. This type of usage record MAY contain any of the base properties. This type of usage record MUST contain at least one of the following properties:o LocalJobId o GlobalJobId" - Aggregate Accounting is specifically allowed, and integral to the current doc. From Sec.5: "Aggregate accounting reports the accounting data in aggregate (summarized form). Aggregate accounting MAY contain any of base properties listed in this document," From Introduction: "Aggregation: Most sites have indicated that job-level aggregation is sufficient for their current needs. Where sites aggregate or decompose to other levels of usage tracking, provisions are available within the schema to accommodate." From Sec.3: "Base Properties: The following is a list of base properties that define common usage record requirements for both job level and aggregate properties." - The Global definitions in sections 7,8 and 9 clearly intend to setup a generic framework in which most types of usage can can be accounted. + Sec.7 sets-up Global Elements to allow substitution for inclusion into other schemas. + Sec.8. sets-up Global attributes sets up generic attributes which maintain meaning independent of context. i.e, forcing common usage of units. + Sec.9. specifically sets-up a *generic* usage record From Sec.7 "The global definitions are those that may be used compliantly with this specification as root elements for an XML document or which may be used as extension points within or included within other XML Schema definitions." From Sec.8 "Global attributes, and attribute groups, are attributes that are common to many of the element definitions contained in this specification. A global definition of these maintains standard semantic meaning within each context that they are used." From Sec.9 "UsageRecordType Complex Type: This complex type definition establishes the structure of the *generic* usage record." - Section 10, Usage Record Common Properties is an obivious attempt to generalise the base proprieties of section 3. The problem is that not all the base properties of section three are appropriate as Common Properties for a *generic* record. Further as the bulk of section 3 is seemingly based on batch level reporting, the scope of the section is serverely limited. From Sec.10 "10. Usage Record Common Properties: the complete specifications for each element that MAY appear in the *generic* usage record are presented below." > > Stick with using email for resolving issues; not worth changing > technologies half way through. > > Short-term Timeline: > Mailing list discussions to close: 30 June > Consensus: 15 July > Revised document: 15 August > Submit to editor: 1 September > > Three-way joint meeting: OGSA RUS UR > > Call for use cases and reference implementations > > Telecons > ======== > Keep going. Biweekly. One hour. 1300 UTC. First on 25 May. > ---------------------------------- > > Donal Fellows. > >

2 1

UR-WG GGF17 Slides (draft) (fwd)
by Laura F. McGinnis 10 May '06

10 May '06

Here are the slides for tomorrow's UR-WG session. Please let me know if I'm missing anything. Because of time constraints we will not be discussing the individual comments at the session. We will focus on process and planning instead. LM

1 0

Aggregate Accounting
by Matthew Ford 09 May '06

09 May '06

Hi All, I hope you have (or have had) a good face-to-face meeting. On the conference call previously we have discussed aggregate accounting and made moves to move it out of scope. I attach below, a valid and well formed record, using the definitions from section 10 of the spec. <UsageRecord> <RecordId urwg:recordId="foo"/> <UserIdentity> <ds:KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#" xmlns:ds="http://www.w3.org/2000/09/xmldsig#"> <X509Data> <X509SubjectName>cn=matt ford</X509SubjectName> </X509Data> </ds:KeyInfo> </UserIdentity> <StartTime urwg:description="accountperiod">2003-06-16T08:24:32Z</StartTime> <EndTime urwg:description="accountperiod">2006-05-08T10:34:58Z</EndTime> <CPUDuration urwg:description="sum over accountperiod for UserIdentity">234234324325</CPUDuration> <WallDuration urwg:description="sum over accountperiod for UserIdentity">23423434</WallDuration> <Status urwg:description="all complete jobs over accounting period for UserIdentity">completed</Status> </UsageRecord> This I think certainly constitutes a valid aggregate record. It would require a farily significant rewrite to make this _not_ be allowed. I'm thinking of using something like this to report total cpuduration for some of my users. Is this against the spirit of the clarification we are trying to make, should this be something that is allowed? The move away from aggregation is a big one...or have I been too liberal in my interpretation (but I'd argue against this) Matt.

2 2

UR-WG at GGF17 in Tokyo
by Laura F McGinnis 09 May '06

09 May '06

The Usage Records Working Group will have one face-to-face session at GGF17 in Tokyo. We are currently scheduled as follows: Thursday, 11 May, 11am-12:30 Room G401 See you there! LM ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Laura F. McGinnis, Project Coordinator Data & Information Resource Services Pittsburgh Supercomputing Center email: lfm(a)psc.edu 300 South Craig Street, #313 voice: 412-268-5642 Pittsburgh, PA 15213 fax: 412-268-8200 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

2 1

Usage vs Job accounting and the UR framework
by Matthew Ford 05 May '06

05 May '06

Donal Wrote: >This point has two distinct aspects. > 1) The StartTime and EndTime values refer to The time period to which > the Usage Record itself applies. For a job (whether grid, batch or > flipping hamburgers for minimum wage) one natural interpretation is > that the start and end indicate an accounting period that matches > when the job is "consuming resources" in some appropriate sense. But > strictly the start and end indicate the accounting period, not the > job. Whether this is true or not depends on which part of the spec you read! Compare 10.10 to 3.13. The former defines StartTime in terms of a "job" and the latter defines in terms of "usage". It is the lack of a clear definition of a "job" and the intermingling of "job" and "usage" in the definitions that are leading, I think, to a *lot* of confusion with the schema. I use job from now on to mean _batch system job_. It goes to the heart of the framework. Regarding the framework ----------------------- Donal's email of the 27th noted the confusion of UsageRecord and JobUsageRecord and talks about the dividing of terms(elements) into generic and computational groups. The division of elements is alluded to in the current spec: base and job level accounting - see the first sentence of Secs. 3 and 4 respectively. It just doesn't seem to be consistently applied. Further, Donal's composite suggestion, also sort-of exists in the current spec: the UsageRecords element (note: it does not allow Sven's nested requirement) - see section 7, Global Element Definitions. Below is a summary of what we have now: Record A: <UsageRecord> <Base Properties..../> </UsageRecord> Record B: <JobUsageRecord> <Base Properties..../> </JobUsageRecord> Record C: <UsageRecords> <UsageRecord> </UsageRecord> ... <JobUsageRecord> </JobUsageRecord> ... </UsageRecords> Records A and B are at the moment equivalent. Record C illustrates the composite structure. It is a *syntactical change of name only*, between Records A and B. The issue is that the Base properties in records A and B are an identical set. The JobUsageRecord whilst inheriting the UsageRecord type does not extend it in anyway (but this may be intentional, I would like to understand the original authors original intent at this point - see questions at end). Without the extension or the separation of the current base properties there is little point to having the two elements UsageRecord and JobUsageRecord. Personally I would like to see the spec use a combination of types and sub-elements to capture usage and to split the current base elements accordingly. I second Donal's proposal. For example; <UsageRecord> <Base Properties> <JobUsageRecord> <CPUDuration/> <Processors/> </JobUsageRecord> </UsageRecord> This allows consistency of data format through types and extensibility through additional elements. It will provide a sensible place for future work on aggregation and allow grid-jobs for example to be much more easily defined. For example: <UsageRecord> <Base Properties> <GlobalJobId/> <Accounting Period/> <Aggregate Info/> </Base Properties> <Grid Job> <JobUsageRecords> <JobUsageRecord/> ... </JobUsageRecords> <DataRecord/> </Grid Job> </UsageRecord> I'm also painfully aware that such a change will mean changes/breaks for existing accounting solutions in place. For example the NGS and it's current RUS service....it may be too big an ask given the current spec. But I believe it's necessary in order to make the spec something more that job accounting. Further it is not feasible for a small group like this to define the elements such as DataRecord and Grid Job (we have a head start on JobUsage). I think the schema should be made extensible to allow other groups to "plug-in" to this structure and that the group should aim to act somehow in a arbitrators role. Notes ----- Historical Questions: Is it that the UsageRecordType should be inherited by elements such as JobUsage or, hypothetically DataUsage? And where possible appropriate elements are mapped onto the global UsageRecord type - this would ensure consistency of data (Duration fields for example through-out the record) but require a certain amount of shoe horning (and potentially a very large global type) to make things work. Why was this approach taken? The extensibility talked about in section 7 - how was that envisaged working? Thanks, Matt. -- Grid Support, Manchester Computing.

1 1