John, I always come back to time integration because once the record is written it cannot be misinterpreted; it is by definition correct even if it is not accurate [that is important]. I understand the inaccuracies of sampling, but the rules of said sampling are specific to the resource, are probably restricted by tooling and must ultimately form part of their service description.
Once one has a collection of records then different grids, projects, infrastructures may apply different procedures.
It is precisely this variation that I am trying to eliminate. Resource consumers and providers are the primary entities concerned with generation and consumption of URs. Grids, brokering systems, infrastructures need not ever be present and as such may make only secondary use of these records, if permitted. It should certainly not be these secondary relying parties that interpret these data with assumptions, for instance, on how regularly a record was cut. <deeper> If you insist on having usage records only define an instantaneous value then it becomes difficult to understand how much the resource was used: You then have 3 choices to retrieve useful information: 1, to requires an infinite set of instantaneous URs [clearly stupid] 2, to gain the set of all URs produced and assumes static between each [requires complex safeguards], 3, rely on the grid infrastructure, that it has some universal way of interpreting this [wrong domain: assumes infrastructure policy, e.g. set by EGI not OGF]. In conclusion, UR must be able to cope with both sampled data and known (continuous) data usage. It must be expressed within the usage record so it cannot in itself be taken out of context. It must therefore contain integrated usage data. I have, up until now, attempted realise what I think you require as incidental information extra to usage but I am rapidly coming to the conclusion that what you really want is an allocation record not a usage record. An allocation record would allow you to subsequently generate a usage record given a set of sampling rules. Mike On Tuesday 21 February 2012 14:07:07 john.gordon@stfc.ac.uk wrote:
Mike, you keep coming back to time integration. I think that once we decided on sampling rather than evaluating integrals then all we have is a series of observations and there are a variety of conclusions we can draw from them. None of them contain the word ‘always’ since we can never know what happens between observations.
Once one has a collection of records then different grids, projects, infrastructures may apply different procedures. I am not against the durationtime/elapsedtime argument but I won’t be attempting to construct time integrals from it. I will not be treating periods of time not covered by records as unused or empty space. I will be calculating min/max/mean over a time period with perhaps some weighting if the end-user projects or sites feel that this isn’t the full story. I might come back and propose a summary UR for passing this information around. I think that for many people desperate for storage accounting these simple metrics will be enough. If they are not then we can reconvene and change/extend the UR.
John
From: Mike Jones via mobile [mailto:mike.jones@manchester.ac.uk] Sent: 21 February 2012 10:56 To: Gordon, John (STFC,RAL,ESC); andrea.cristofori@cnaf.infn.it; j.k.nilsen@fys.uio.no Cc: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
To follow up on John's WLCG usecase: I agree that VO space allocated over a period of time _is_ a usage. BUT VO allocation at any instant is not. That said, the dynamics of any allocation/usage needs to be recorded to be able to answer the question "over period X did the allocation always satisfy: instantaneous_storage( used+available) > N (dimension:bytes)?" John, does that last clause satisfy WLCG?
Mike -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. john.gordon@stfc.ac.uk wrote: I’ll be in Taiwan next week UTC+8 so I may or may not attend. Depends if anything official is happening that evening and if I can stay awake and if Skype works.
I’ll add the WLCG storage use case to the wiki before then. I must stress that this is important to WLCG and that we consider space allocated to a VO as ‘usage’ as it is not available to others and is to be recorded just like space occupied by files. i.e. Not monitoring any more than recording the existence of files in monitoring.
John
From: ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] On Behalf Of Andrea Cristofori Sent: 21 February 2012 08:15 To: Jon Kerr Nilsen Cc: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
Dear all,
Sorry for the short notice but neither I or Jon can make it to today phone conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF:
http://www.ogf.org/gf/event_schedule/index.php?id=2413
Again apologies.
Cheers, Andrea
On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote: Hi all,
As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to
Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608
cheers, Jon
On 7. feb. 2012, at 17:32, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases:
1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have. All: seems to be more monitoring Comments from John are required
1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB). All: seems to be more monitoring Comments from John are required
Cheers Andrea -- ur-wg mailing list ur-wg@ogf.org<mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg
--
ur-wg mailing list
ur-wg@ogf.org<mailto:ur-wg@ogf.org>
https://www.ogf.org/mailman/listinfo/ur-wg
--
Andrea Cristofori
INFN-CNAF
Viale Berti Pichat 6/2
40127 Bologna
Italy
Tel. : +39-051-6092920
Skype: andrea-cnaf
-- Scanned by iCritical.