Actions for next phone meeting on 21.02.2012
Dear all, Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following: - AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache) Especially for John Gordon: can you please comment on those two use cases: 1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have. All: seems to be more monitoring Comments from John are required 1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB). All: seems to be more monitoring Comments from John are required Cheers Andrea
Dear all, Here is our scheduled session (90 minutes) for next OGF: https://www.egi.eu/indico/conferenceDisplay.py?confId=821 Cheers, Andrea On 02/07/2012 05:32 PM, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases:
1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have.
All: seems to be more monitoring Comments from John are required
1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB).
All: seems to be more monitoring Comments from John are required
Cheers Andrea
-- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg
-- Andrea Cristofori INFN-CNAF Viale Berti Pichat 6/2 40127 Bologna Italy Tel. : +39-051-6092920 Skype: andrea-cnaf
Andrea, are you sure you sent the correct link? From: ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] On Behalf Of Andrea Cristofori Sent: 20 February 2012 17:08 To: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012 Dear all, Here is our scheduled session (90 minutes) for next OGF: https://www.egi.eu/indico/conferenceDisplay.py?confId=821 Cheers, Andrea On 02/07/2012 05:32 PM, Andrea Cristofori wrote: Dear all, Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following: - AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache) Especially for John Gordon: can you please comment on those two use cases: 1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have. All: seems to be more monitoring Comments from John are required 1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB). All: seems to be more monitoring Comments from John are required Cheers Andrea -- ur-wg mailing list ur-wg@ogf.org<mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg -- Andrea Cristofori INFN-CNAF Viale Berti Pichat 6/2 40127 Bologna Italy Tel. : +39-051-6092920 Skype: andrea-cnaf -- Scanned by iCritical.
Hi John, I was in a hurry... sorry wrong link. Here is the correct one: http://www.ogf.org/gf/event_schedule/index.php?id=2413 Andrea On 02/20/2012 06:22 PM, john.gordon@stfc.ac.uk wrote:
Andrea, are you sure you sent the correct link?
*From:*ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] *On Behalf Of *Andrea Cristofori *Sent:* 20 February 2012 17:08 *To:* ur-wg@ogf.org *Subject:* Re: [UR-WG] Actions for next phone meeting on 21.02.2012
Dear all,
Here is our scheduled session (90 minutes) for next OGF:
https://www.egi.eu/indico/conferenceDisplay.py?confId=821
Cheers, Andrea
On 02/07/2012 05:32 PM, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases:
1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have.
All: seems to be more monitoring Comments from John are required
1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB).
All: seems to be more monitoring Comments from John are required
Cheers Andrea
-- ur-wg mailing list ur-wg@ogf.org <mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg
-- Andrea Cristofori INFN-CNAF Viale Berti Pichat 6/2 40127 Bologna Italy Tel. : +39-051-6092920 Skype: andrea-cnaf
-- Scanned by iCritical.
-- Andrea Cristofori INFN-CNAF Viale Berti Pichat 6/2 40127 Bologna Italy Tel. : +39-051-6092920 Skype: andrea-cnaf
Hi Andrea, I am sorry but I can't make the phone meeting today. I'll try to do my homework and send an example grid record - I should learn something by doing this anyway. cheers johnk On 02/20/2012 06:07 PM, Andrea Cristofori wrote:
Dear all,
Here is our scheduled session (90 minutes) for next OGF:
https://www.egi.eu/indico/conferenceDisplay.py?confId=821
Cheers, Andrea
On 02/07/2012 05:32 PM, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases:
1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have.
All: seems to be more monitoring Comments from John are required
1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB).
All: seems to be more monitoring Comments from John are required
Cheers Andrea
-- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg
-- Andrea Cristofori INFN-CNAF Viale Berti Pichat 6/2 40127 Bologna Italy Tel. : +39-051-6092920 Skype: andrea-cnaf
-- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg
-- +------------------------------------------------------------+ |Dr. John Alan Kennedy Rechenzentrum Garching (RZG) | |Mail: jkennedy@rzg.mpg.de Boltzmannstrasse 2 | |Phone: +49 89 3299 2694 85748 Garching | |Fax: +49 89 3299 1301 | +------------------------------------------------------------+
Hi all, As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608 cheers, Jon On 7. feb. 2012, at 17:32, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases:
As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have. All: seems to be more monitoring Comments from John are required As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB). All: seems to be more monitoring Comments from John are required
Cheers Andrea -- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg
Dear all, Sorry for the short notice but neither I or Jon can make it to today phone conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF: http://www.ogf.org/gf/event_schedule/index.php?id=2413 Again apologies. Cheers, Andrea On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote:
Hi all,
As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to
Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608
cheers, Jon
On 7. feb. 2012, at 17:32, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases:
1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have.
All: seems to be more monitoring Comments from John are required
1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB).
All: seems to be more monitoring Comments from John are required
Cheers Andrea -- ur-wg mailing list ur-wg@ogf.org <mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg
-- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg
-- Andrea Cristofori INFN-CNAF Viale Berti Pichat 6/2 40127 Bologna Italy Tel. : +39-051-6092920 Skype: andrea-cnaf
Hi, next Tuesday is fine for me. Cheers, Ralph Andrea Cristofori wrote on 21.02.2012 09:15:
Dear all,
Sorry for the short notice but neither I or Jon can make it to today phone conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF:
http://www.ogf.org/gf/event_schedule/index.php?id=2413
Again apologies.
Cheers, Andrea
On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote:
Hi all,
As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to
Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608
cheers, Jon
On 7. feb. 2012, at 17:32, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases:
1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have.
All: seems to be more monitoring Comments from John are required
1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB).
All: seems to be more monitoring Comments from John are required
Cheers Andrea -- ur-wg mailing list ur-wg@ogf.org <mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg
-- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg
-- Andrea Cristofori INFN-CNAF Viale Berti Pichat 6/2 40127 Bologna Italy Tel. : +39-051-6092920 Skype: andrea-cnaf
-- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg
I'll be in Taiwan next week UTC+8 so I may or may not attend. Depends if anything official is happening that evening and if I can stay awake and if Skype works. I'll add the WLCG storage use case to the wiki before then. I must stress that this is important to WLCG and that we consider space allocated to a VO as 'usage' as it is not available to others and is to be recorded just like space occupied by files. i.e. Not monitoring any more than recording the existence of files in monitoring. John From: ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] On Behalf Of Andrea Cristofori Sent: 21 February 2012 08:15 To: Jon Kerr Nilsen Cc: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET Dear all, Sorry for the short notice but neither I or Jon can make it to today phone conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF: http://www.ogf.org/gf/event_schedule/index.php?id=2413 Again apologies. Cheers, Andrea On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote: Hi all, As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608 cheers, Jon On 7. feb. 2012, at 17:32, Andrea Cristofori wrote: Dear all, Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following: - AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache) Especially for John Gordon: can you please comment on those two use cases: 1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have. All: seems to be more monitoring Comments from John are required 1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB). All: seems to be more monitoring Comments from John are required Cheers Andrea -- ur-wg mailing list ur-wg@ogf.org<mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg -- ur-wg mailing list ur-wg@ogf.org<mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg -- Andrea Cristofori INFN-CNAF Viale Berti Pichat 6/2 40127 Bologna Italy Tel. : +39-051-6092920 Skype: andrea-cnaf -- Scanned by iCritical.
To follow up on John's WLCG usecase: I agree that VO space allocated over a period of time _is_ a usage. BUT VO allocation at any instant is not. That said, the dynamics of any allocation/usage needs to be recorded to be able to answer the question "over period X did the allocation always satisfy: instantaneous_storage( used+available) > N (dimension:bytes)?" John, does that last clause satisfy WLCG? Mike -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. john.gordon@stfc.ac.uk wrote: I’ll be in Taiwan next week UTC+8 so I may or may not attend. Depends if anything official is happening that evening and if I can stay awake and if Skype works. I’ll add the WLCG storage use case to the wiki before then. I must stress that this is important to WLCG and that we consider space allocated to a VO as ‘usage’ as it is not available to others and is to be recorded just like space occupied by files. i.e. Not monitoring any more than recording the existence of files in monitoring. John From: ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] On Behalf Of Andrea Cristofori Sent: 21 February 2012 08:15 To: Jon Kerr Nilsen Cc: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET Dear all, Sorry for the short notice but neither I or Jon can make it to today phone conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF: http://www.ogf.org/gf/event_schedule/index.php?id=2413 Again apologies. Cheers, Andrea On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote: Hi all, As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608 cheers, Jon On 7. feb. 2012, at 17:32, Andrea Cristofori wrote: Dear all, Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following: - AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache) Especially for John Gordon: can you please comment on those two use cases: As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have. All: seems to be more monitoring Comments from John are required As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB). All: seems to be more monitoring Comments from John are required Cheers Andrea -- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg -- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg -- Andrea Cristofori INFN-CNAF Viale Berti Pichat 6/2 40127 Bologna Italy Tel. : +39-051-6092920 Skype: andrea-cnaf -- Scanned by iCritical.
Hi, Mike Jones via mobile wrote on 21.02.2012 11:55:
To follow up on John's WLCG usecase: I agree that VO space allocated over a period of time _is_ a usage. BUT VO allocation at any instant is not. That said, the dynamics of any allocation/usage needs to be recorded to be able to answer the question "over period X did the allocation always satisfy: instantaneous_storage( used+available) > N (dimension:bytes)?" A possible implementation could be to have records for the usage and records stating the allocated space such that it is possible to do such a comparison. E.g. (in the StAR scheme)
Allocation: ... <ur:StorageSystem>host.example.org</ur:StorageSystem> <ur:StorageShare>pool-003</ur:StorageShare> <ur:StorageClass>allocated</ur:StorageClass> <ur:StorageResourceCapacityUsed>14728</ur:StorageResourceCapacityUsed> ... Usage: ... <ur:StorageSystem>host.example.org</ur:StorageSystem> <ur:StorageShare>pool-003</ur:StorageShare> <ur:StorageResourceCapacityUsed>1728</ur:StorageResourceCapacityUsed> ... This would mean that the analysis of the records has to be careful in not mixing the different meanings of the records. Cheers, Ralph
John, does that last clause satisfy WLCG?
Mike -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
john.gordon@stfc.ac.uk wrote:
I’ll be in Taiwan next week UTC+8 so I may or may not attend. Depends if anything official is happening that evening and if I can stay awake and if Skype works.
I’ll add the WLCG storage use case to the wiki before then. I must stress that this is important to WLCG and that we consider space allocated to a VO as ‘usage’ as it is not available to others and is to be recorded just like space occupied by files. i.e. Not monitoring any more than recording the existence of files in monitoring.
John
*From:*ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] *On Behalf Of *Andrea Cristofori *Sent:* 21 February 2012 08:15 *To:* Jon Kerr Nilsen *Cc:* ur-wg@ogf.org *Subject:* Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
Dear all,
Sorry for the short notice but neither I or Jon can make it to today phone conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF:
http://www.ogf.org/gf/event_schedule/index.php?id=2413
Again apologies.
Cheers, Andrea
On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote:
Hi all,
As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to
Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608
cheers,
Jon
On 7. feb. 2012, at 17:32, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases:
1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have.
All: seems to be more monitoring Comments from John are required
1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB).
All: seems to be more monitoring Comments from John are required
Cheers Andrea
-- ur-wg mailing list ur-wg@ogf.org <mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg
--
ur-wg mailing list
ur-wg@ogf.org <mailto:ur-wg@ogf.org>
https://www.ogf.org/mailman/listinfo/ur-wg
--
Andrea Cristofori
INFN-CNAF
Viale Berti Pichat 6/2
40127 Bologna
Italy
Tel. :+39-051-6092920
Skype: andrea-cnaf
-- Scanned by iCritical.
-- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg
Just got a use case for storage monitoring (not accounting): the OGF Wiki server. Wanted to add something and got: Error saving wiki page: No space left on device :) Cheers, Ralph Ralph Mueller-Pfefferkorn wrote on 21.02.2012 14:15:
Hi,
Mike Jones via mobile wrote on 21.02.2012 11:55:
To follow up on John's WLCG usecase: I agree that VO space allocated over a period of time _is_ a usage. BUT VO allocation at any instant is not. That said, the dynamics of any allocation/usage needs to be recorded to be able to answer the question "over period X did the allocation always satisfy: instantaneous_storage( used+available)> N (dimension:bytes)?" A possible implementation could be to have records for the usage and records stating the allocated space such that it is possible to do such a comparison. E.g. (in the StAR scheme)
Allocation: ... <ur:StorageSystem>host.example.org</ur:StorageSystem> <ur:StorageShare>pool-003</ur:StorageShare> <ur:StorageClass>allocated</ur:StorageClass> <ur:StorageResourceCapacityUsed>14728</ur:StorageResourceCapacityUsed> ...
Usage: ... <ur:StorageSystem>host.example.org</ur:StorageSystem> <ur:StorageShare>pool-003</ur:StorageShare> <ur:StorageResourceCapacityUsed>1728</ur:StorageResourceCapacityUsed> ...
This would mean that the analysis of the records has to be careful in not mixing the different meanings of the records.
Cheers, Ralph
John, does that last clause satisfy WLCG?
Mike -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
john.gordon@stfc.ac.uk wrote:
I’ll be in Taiwan next week UTC+8 so I may or may not attend. Depends if anything official is happening that evening and if I can stay awake and if Skype works.
I’ll add the WLCG storage use case to the wiki before then. I must stress that this is important to WLCG and that we consider space allocated to a VO as ‘usage’ as it is not available to others and is to be recorded just like space occupied by files. i.e. Not monitoring any more than recording the existence of files in monitoring.
John
*From:*ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] *On Behalf Of *Andrea Cristofori *Sent:* 21 February 2012 08:15 *To:* Jon Kerr Nilsen *Cc:* ur-wg@ogf.org *Subject:* Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
Dear all,
Sorry for the short notice but neither I or Jon can make it to today phone conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF:
http://www.ogf.org/gf/event_schedule/index.php?id=2413
Again apologies.
Cheers, Andrea
On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote:
Hi all,
As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to
Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608
cheers,
Jon
On 7. feb. 2012, at 17:32, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases:
1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have.
All: seems to be more monitoring Comments from John are required
1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB).
All: seems to be more monitoring Comments from John are required
Cheers Andrea
-- ur-wg mailing list ur-wg@ogf.org<mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg
--
ur-wg mailing list
ur-wg@ogf.org<mailto:ur-wg@ogf.org>
https://www.ogf.org/mailman/listinfo/ur-wg
--
Andrea Cristofori
INFN-CNAF
Viale Berti Pichat 6/2
40127 Bologna
Italy
Tel. :+39-051-6092920
Skype: andrea-cnaf
-- Scanned by iCritical.
-- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg
-- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg
The risk implied by your 'careful' is why I prefer both quantities to be in the same UR. Then each observation is unique and one needn't mess around trying to reconcile time differences and their implications. John
-----Original Message----- From: ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] On Behalf Of Ralph Mueller-Pfefferkorn Sent: 21 February 2012 13:16 Cc: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
Hi,
Mike Jones via mobile wrote on 21.02.2012 11:55:
To follow up on John's WLCG usecase: I agree that VO space allocated over a period of time _is_ a usage. BUT VO allocation at any instant is not. That said, the dynamics of any allocation/usage needs to be recorded to be able to answer the question "over period X did the allocation always satisfy: instantaneous_storage( used+available) > N (dimension:bytes)?" A possible implementation could be to have records for the usage and records stating the allocated space such that it is possible to do such a comparison. E.g. (in the StAR scheme)
Allocation: ... <ur:StorageSystem>host.example.org</ur:StorageSystem> <ur:StorageShare>pool-003</ur:StorageShare> <ur:StorageClass>allocated</ur:StorageClass> <ur:StorageResourceCapacityUsed>14728</ur:StorageResourceCapacityUsed> ...
Usage: ... <ur:StorageSystem>host.example.org</ur:StorageSystem> <ur:StorageShare>pool-003</ur:StorageShare> <ur:StorageResourceCapacityUsed>1728</ur:StorageResourceCapacityUsed> ...
This would mean that the analysis of the records has to be careful in not mixing the different meanings of the records.
Cheers, Ralph
John, does that last clause satisfy WLCG?
Mike -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
john.gordon@stfc.ac.uk wrote:
I’ll be in Taiwan next week UTC+8 so I may or may not attend. Depends if anything official is happening that evening and if I can stay awake and if Skype works.
I’ll add the WLCG storage use case to the wiki before then. I must stress that this is important to WLCG and that we consider space allocated to a VO as ‘usage’ as it is not available to others and is to be recorded just like space occupied by files. i.e. Not monitoring any more than recording the existence of files in monitoring.
John
*From:*ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] *On Behalf Of *Andrea Cristofori *Sent:* 21 February 2012 08:15 *To:* Jon Kerr Nilsen *Cc:* ur-wg@ogf.org *Subject:* Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
Dear all,
Sorry for the short notice but neither I or Jon can make it to today phone conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF:
http://www.ogf.org/gf/event_schedule/index.php?id=2413
Again apologies.
Cheers, Andrea
On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote:
Hi all,
As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to
Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608
cheers,
Jon
On 7. feb. 2012, at 17:32, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases:
1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have.
All: seems to be more monitoring Comments from John are required
1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB).
All: seems to be more monitoring Comments from John are required
Cheers Andrea
-- ur-wg mailing list ur-wg@ogf.org <mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg
--
ur-wg mailing list
ur-wg@ogf.org <mailto:ur-wg@ogf.org>
https://www.ogf.org/mailman/listinfo/ur-wg
--
Andrea Cristofori
INFN-CNAF
Viale Berti Pichat 6/2
40127 Bologna
Italy
Tel. :+39-051-6092920
Skype: andrea-cnaf
-- Scanned by iCritical.
-- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg
Mike, you keep coming back to time integration. I think that once we decided on sampling rather than evaluating integrals then all we have is a series of observations and there are a variety of conclusions we can draw from them. None of them contain the word ‘always’ since we can never know what happens between observations. Once one has a collection of records then different grids, projects, infrastructures may apply different procedures. I am not against the durationtime/elapsedtime argument but I won’t be attempting to construct time integrals from it. I will not be treating periods of time not covered by records as unused or empty space. I will be calculating min/max/mean over a time period with perhaps some weighting if the end-user projects or sites feel that this isn’t the full story. I might come back and propose a summary UR for passing this information around. I think that for many people desperate for storage accounting these simple metrics will be enough. If they are not then we can reconvene and change/extend the UR. John From: Mike Jones via mobile [mailto:mike.jones@manchester.ac.uk] Sent: 21 February 2012 10:56 To: Gordon, John (STFC,RAL,ESC); andrea.cristofori@cnaf.infn.it; j.k.nilsen@fys.uio.no Cc: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET To follow up on John's WLCG usecase: I agree that VO space allocated over a period of time _is_ a usage. BUT VO allocation at any instant is not. That said, the dynamics of any allocation/usage needs to be recorded to be able to answer the question "over period X did the allocation always satisfy: instantaneous_storage( used+available) > N (dimension:bytes)?" John, does that last clause satisfy WLCG? Mike -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. john.gordon@stfc.ac.uk wrote: I’ll be in Taiwan next week UTC+8 so I may or may not attend. Depends if anything official is happening that evening and if I can stay awake and if Skype works. I’ll add the WLCG storage use case to the wiki before then. I must stress that this is important to WLCG and that we consider space allocated to a VO as ‘usage’ as it is not available to others and is to be recorded just like space occupied by files. i.e. Not monitoring any more than recording the existence of files in monitoring. John From: ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] On Behalf Of Andrea Cristofori Sent: 21 February 2012 08:15 To: Jon Kerr Nilsen Cc: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET Dear all, Sorry for the short notice but neither I or Jon can make it to today phone conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF: http://www.ogf.org/gf/event_schedule/index.php?id=2413 Again apologies. Cheers, Andrea On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote: Hi all, As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608 cheers, Jon On 7. feb. 2012, at 17:32, Andrea Cristofori wrote: Dear all, Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following: - AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache) Especially for John Gordon: can you please comment on those two use cases: 1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have. All: seems to be more monitoring Comments from John are required 1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB). All: seems to be more monitoring Comments from John are required Cheers Andrea -- ur-wg mailing list ur-wg@ogf.org<mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg -- ur-wg mailing list ur-wg@ogf.org<mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg -- Andrea Cristofori INFN-CNAF Viale Berti Pichat 6/2 40127 Bologna Italy Tel. : +39-051-6092920 Skype: andrea-cnaf -- Scanned by iCritical.
John, I always come back to time integration because once the record is written it cannot be misinterpreted; it is by definition correct even if it is not accurate [that is important]. I understand the inaccuracies of sampling, but the rules of said sampling are specific to the resource, are probably restricted by tooling and must ultimately form part of their service description.
Once one has a collection of records then different grids, projects, infrastructures may apply different procedures.
It is precisely this variation that I am trying to eliminate. Resource consumers and providers are the primary entities concerned with generation and consumption of URs. Grids, brokering systems, infrastructures need not ever be present and as such may make only secondary use of these records, if permitted. It should certainly not be these secondary relying parties that interpret these data with assumptions, for instance, on how regularly a record was cut. <deeper> If you insist on having usage records only define an instantaneous value then it becomes difficult to understand how much the resource was used: You then have 3 choices to retrieve useful information: 1, to requires an infinite set of instantaneous URs [clearly stupid] 2, to gain the set of all URs produced and assumes static between each [requires complex safeguards], 3, rely on the grid infrastructure, that it has some universal way of interpreting this [wrong domain: assumes infrastructure policy, e.g. set by EGI not OGF]. In conclusion, UR must be able to cope with both sampled data and known (continuous) data usage. It must be expressed within the usage record so it cannot in itself be taken out of context. It must therefore contain integrated usage data. I have, up until now, attempted realise what I think you require as incidental information extra to usage but I am rapidly coming to the conclusion that what you really want is an allocation record not a usage record. An allocation record would allow you to subsequently generate a usage record given a set of sampling rules. Mike On Tuesday 21 February 2012 14:07:07 john.gordon@stfc.ac.uk wrote:
Mike, you keep coming back to time integration. I think that once we decided on sampling rather than evaluating integrals then all we have is a series of observations and there are a variety of conclusions we can draw from them. None of them contain the word ‘always’ since we can never know what happens between observations.
Once one has a collection of records then different grids, projects, infrastructures may apply different procedures. I am not against the durationtime/elapsedtime argument but I won’t be attempting to construct time integrals from it. I will not be treating periods of time not covered by records as unused or empty space. I will be calculating min/max/mean over a time period with perhaps some weighting if the end-user projects or sites feel that this isn’t the full story. I might come back and propose a summary UR for passing this information around. I think that for many people desperate for storage accounting these simple metrics will be enough. If they are not then we can reconvene and change/extend the UR.
John
From: Mike Jones via mobile [mailto:mike.jones@manchester.ac.uk] Sent: 21 February 2012 10:56 To: Gordon, John (STFC,RAL,ESC); andrea.cristofori@cnaf.infn.it; j.k.nilsen@fys.uio.no Cc: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
To follow up on John's WLCG usecase: I agree that VO space allocated over a period of time _is_ a usage. BUT VO allocation at any instant is not. That said, the dynamics of any allocation/usage needs to be recorded to be able to answer the question "over period X did the allocation always satisfy: instantaneous_storage( used+available) > N (dimension:bytes)?" John, does that last clause satisfy WLCG?
Mike -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. john.gordon@stfc.ac.uk wrote: I’ll be in Taiwan next week UTC+8 so I may or may not attend. Depends if anything official is happening that evening and if I can stay awake and if Skype works.
I’ll add the WLCG storage use case to the wiki before then. I must stress that this is important to WLCG and that we consider space allocated to a VO as ‘usage’ as it is not available to others and is to be recorded just like space occupied by files. i.e. Not monitoring any more than recording the existence of files in monitoring.
John
From: ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] On Behalf Of Andrea Cristofori Sent: 21 February 2012 08:15 To: Jon Kerr Nilsen Cc: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
Dear all,
Sorry for the short notice but neither I or Jon can make it to today phone conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF:
http://www.ogf.org/gf/event_schedule/index.php?id=2413
Again apologies.
Cheers, Andrea
On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote: Hi all,
As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to
Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608
cheers, Jon
On 7. feb. 2012, at 17:32, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases:
1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have. All: seems to be more monitoring Comments from John are required
1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB). All: seems to be more monitoring Comments from John are required
Cheers Andrea -- ur-wg mailing list ur-wg@ogf.org<mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg
--
ur-wg mailing list
ur-wg@ogf.org<mailto:ur-wg@ogf.org>
https://www.ogf.org/mailman/listinfo/ur-wg
--
Andrea Cristofori
INFN-CNAF
Viale Berti Pichat 6/2
40127 Bologna
Italy
Tel. : +39-051-6092920
Skype: andrea-cnaf
-- Scanned by iCritical.
Mike, I think you are way out of line here. As long ago as Munich OGF we discussed time integrals for UR and I thought we reached a consensus (well maybe everyone except you) that we could not ask storage systems to start evaluating integrals of storage usage for the use of UR. They don't do it and we would be asking them to implement journaling or to recalculate usage at every file interaction (or at least write/modify/delete). Just not on. I thought we all understood that this is a compromise for the pragmatic purpose of getting a UR soon that can be implemented in the real world. You seem to enjoy the tautological discussion of what accounting should be in the theoretical world. I just want to start writing records tomorrow, if not today. In the longer term maybe we make requirements against storage systems that they implement integrals as well as user level and i/o accounting. For now can we concentrate on agreeing something that can be implemented. John PS I'll try to call in. I'm in Taipei.
-----Original Message----- From: Mike Jones [mailto:mike.jones@manchester.ac.uk] Sent: 21 February 2012 18:48 To: Gordon, John (STFC,RAL,ESC) Cc: andrea.cristofori@cnaf.infn.it; j.k.nilsen@fys.uio.no; ur- wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
John,
I always come back to time integration because once the record is written it cannot be misinterpreted; it is by definition correct even if it is not accurate [that is important]. I understand the inaccuracies of sampling, but the rules of said sampling are specific to the resource, are probably restricted by tooling and must ultimately form part of their service description.
Once one has a collection of records then different grids, projects, infrastructures may apply different procedures.
It is precisely this variation that I am trying to eliminate. Resource consumers and providers are the primary entities concerned with generation and consumption of URs. Grids, brokering systems, infrastructures need not ever be present and as such may make only secondary use of these records, if permitted. It should certainly not be these secondary relying parties that interpret these data with assumptions, for instance, on how regularly a record was cut.
<deeper> If you insist on having usage records only define an instantaneous value then it becomes difficult to understand how much the resource was used: You then have 3 choices to retrieve useful information: 1, to requires an infinite set of instantaneous URs [clearly stupid] 2, to gain the set of all URs produced and assumes static between each [requires complex safeguards], 3, rely on the grid infrastructure, that it has some universal way of interpreting this [wrong domain: assumes infrastructure policy, e.g. set by EGI not OGF].
In conclusion, UR must be able to cope with both sampled data and known (continuous) data usage. It must be expressed within the usage record so it cannot in itself be taken out of context. It must therefore contain integrated usage data.
I have, up until now, attempted realise what I think you require as incidental information extra to usage but I am rapidly coming to the conclusion that what you really want is an allocation record not a usage record. An allocation record would allow you to subsequently generate a usage record given a set of sampling rules.
Mike
Mike, you keep coming back to time integration. I think that once we decided on sampling rather than evaluating integrals then all we have is a series of observations and there are a variety of conclusions we can draw from them. None of them contain the word ‘always’ since we can never know what happens between observations.
Once one has a collection of records then different grids, projects, infrastructures may apply different procedures. I am not against the durationtime/elapsedtime argument but I won’t be attempting to construct time integrals from it. I will not be treating periods of time not covered by records as unused or empty space. I will be calculating min/max/mean over a time period with perhaps some weighting if the end-user
sites feel that this isn’t the full story. I might come back and
summary UR for passing this information around. I think that for many people desperate for storage accounting these simple metrics will be enough. If they are not then we can reconvene and change/extend the UR.
John
From: Mike Jones via mobile [mailto:mike.jones@manchester.ac.uk] Sent: 21 February 2012 10:56 To: Gordon, John (STFC,RAL,ESC); andrea.cristofori@cnaf.infn.it; j.k.nilsen@fys.uio.no Cc: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
To follow up on John's WLCG usecase: I agree that VO space allocated over a period of time _is_ a usage. BUT VO allocation at any instant is not. That said, the dynamics of any allocation/usage needs to be recorded to be able to answer the question "over period X did the allocation always satisfy: instantaneous_storage( used+available) > N (dimension:bytes)?" John, does that last clause satisfy WLCG?
Mike -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. john.gordon@stfc.ac.uk wrote: I’ll be in Taiwan next week UTC+8 so I may or may not attend. Depends if anything official is happening that evening and if I can stay awake and if Skype works.
I’ll add the WLCG storage use case to the wiki before then. I must stress that this is important to WLCG and that we consider space allocated to a VO as ‘usage’ as it is not available to others and is to be recorded just like space occupied by files. i.e. Not monitoring any more than recording the existence of files in monitoring.
John
From: ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] On Behalf Of Andrea Cristofori Sent: 21 February 2012 08:15 To: Jon Kerr Nilsen Cc: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
Dear all,
Sorry for the short notice but neither I or Jon can make it to today
conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF:
http://www.ogf.org/gf/event_schedule/index.php?id=2413
Again apologies.
Cheers, Andrea
On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote: Hi all,
As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to
Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608
cheers, Jon
On 7. feb. 2012, at 17:32, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two
On Tuesday 21 February 2012 14:07:07 john.gordon@stfc.ac.uk wrote: projects or propose a phone parts
and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases:
1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have. All: seems to be more monitoring Comments from John are required
1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB). All: seems to be more monitoring Comments from John are required
Cheers Andrea -- ur-wg mailing list ur-wg@ogf.org<mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg
--
ur-wg mailing list
ur-wg@ogf.org<mailto:ur-wg@ogf.org>
https://www.ogf.org/mailman/listinfo/ur-wg
--
Andrea Cristofori
INFN-CNAF
Viale Berti Pichat 6/2
40127 Bologna
Italy
Tel. : +39-051-6092920
Skype: andrea-cnaf
-- Scanned by iCritical.
Hi John, I think there were at least two of us that understood that some sort of integral was needed. In fact after the discussion on integral vs sampling we moved on the discussion of timeduration vs validduration. For me it was now clear that we moved in this direction. This allows a single record to give all the information we need on the storage usage. Andrea On 02/28/2012 10:58 AM, john.gordon@stfc.ac.uk wrote:
Mike, I think you are way out of line here. As long ago as Munich OGF we discussed time integrals for UR and I thought we reached a consensus (well maybe everyone except you) that we could not ask storage systems to start evaluating integrals of storage usage for the use of UR. They don't do it and we would be asking them to implement journaling or to recalculate usage at every file interaction (or at least write/modify/delete). Just not on.
I thought we all understood that this is a compromise for the pragmatic purpose of getting a UR soon that can be implemented in the real world. You seem to enjoy the tautological discussion of what accounting should be in the theoretical world. I just want to start writing records tomorrow, if not today.
In the longer term maybe we make requirements against storage systems that they implement integrals as well as user level and i/o accounting.
For now can we concentrate on agreeing something that can be implemented.
John
PS I'll try to call in. I'm in Taipei.
-----Original Message----- From: Mike Jones [mailto:mike.jones@manchester.ac.uk] Sent: 21 February 2012 18:48 To: Gordon, John (STFC,RAL,ESC) Cc: andrea.cristofori@cnaf.infn.it; j.k.nilsen@fys.uio.no; ur- wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
John,
I always come back to time integration because once the record is written it cannot be misinterpreted; it is by definition correct even if it is not accurate [that is important]. I understand the inaccuracies of sampling, but the rules of said sampling are specific to the resource, are probably restricted by tooling and must ultimately form part of their service description.
Once one has a collection of records then different grids, projects, infrastructures may apply different procedures. It is precisely this variation that I am trying to eliminate. Resource consumers and providers are the primary entities concerned with generation and consumption of URs. Grids, brokering systems, infrastructures need not ever be present and as such may make only secondary use of these records, if permitted. It should certainly not be these secondary relying parties that interpret these data with assumptions, for instance, on how regularly a record was cut.
<deeper> If you insist on having usage records only define an instantaneous value then it becomes difficult to understand how much the resource was used: You then have 3 choices to retrieve useful information: 1, to requires an infinite set of instantaneous URs [clearly stupid] 2, to gain the set of all URs produced and assumes static between each [requires complex safeguards], 3, rely on the grid infrastructure, that it has some universal way of interpreting this [wrong domain: assumes infrastructure policy, e.g. set by EGI not OGF].
In conclusion, UR must be able to cope with both sampled data and known (continuous) data usage. It must be expressed within the usage record so it cannot in itself be taken out of context. It must therefore contain integrated usage data.
I have, up until now, attempted realise what I think you require as incidental information extra to usage but I am rapidly coming to the conclusion that what you really want is an allocation record not a usage record. An allocation record would allow you to subsequently generate a usage record given a set of sampling rules.
Mike
Mike, you keep coming back to time integration. I think that once we decided on sampling rather than evaluating integrals then all we have is a series of observations and there are a variety of conclusions we can draw from them. None of them contain the word ‘always’ since we can never know what happens between observations.
Once one has a collection of records then different grids, projects, infrastructures may apply different procedures. I am not against the durationtime/elapsedtime argument but I won’t be attempting to construct time integrals from it. I will not be treating periods of time not covered by records as unused or empty space. I will be calculating min/max/mean over a time period with perhaps some weighting if the end-user
sites feel that this isn’t the full story. I might come back and
summary UR for passing this information around. I think that for many people desperate for storage accounting these simple metrics will be enough. If they are not then we can reconvene and change/extend the UR. John
From: Mike Jones via mobile [mailto:mike.jones@manchester.ac.uk] Sent: 21 February 2012 10:56 To: Gordon, John (STFC,RAL,ESC); andrea.cristofori@cnaf.infn.it; j.k.nilsen@fys.uio.no Cc: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
To follow up on John's WLCG usecase: I agree that VO space allocated over a period of time _is_ a usage. BUT VO allocation at any instant is not. That said, the dynamics of any allocation/usage needs to be recorded to be able to answer the question "over period X did the allocation always satisfy: instantaneous_storage( used+available)> N (dimension:bytes)?" John, does that last clause satisfy WLCG?
Mike -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. john.gordon@stfc.ac.uk wrote: I’ll be in Taiwan next week UTC+8 so I may or may not attend. Depends if anything official is happening that evening and if I can stay awake and if Skype works.
I’ll add the WLCG storage use case to the wiki before then. I must stress that this is important to WLCG and that we consider space allocated to a VO as ‘usage’ as it is not available to others and is to be recorded just like space occupied by files. i.e. Not monitoring any more than recording the existence of files in monitoring.
John
From: ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] On Behalf Of Andrea Cristofori Sent: 21 February 2012 08:15 To: Jon Kerr Nilsen Cc: ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
Dear all,
Sorry for the short notice but neither I or Jon can make it to today
conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF:
http://www.ogf.org/gf/event_schedule/index.php?id=2413
Again apologies.
Cheers, Andrea
On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote: Hi all,
As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to
Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608
cheers, Jon
On 7. feb. 2012, at 17:32, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two
On Tuesday 21 February 2012 14:07:07 john.gordon@stfc.ac.uk wrote: projects or propose a phone parts
and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases: 1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have. All: seems to be more monitoring Comments from John are required
1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB). All: seems to be more monitoring Comments from John are required
Cheers Andrea -- ur-wg mailing list ur-wg@ogf.org<mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg
--
ur-wg mailing list
ur-wg@ogf.org<mailto:ur-wg@ogf.org>
https://www.ogf.org/mailman/listinfo/ur-wg
--
Andrea Cristofori
INFN-CNAF
Viale Berti Pichat 6/2
40127 Bologna
Italy
Tel. : +39-051-6092920
Skype: andrea-cnaf
-- Scanned by iCritical.
-- Andrea Cristofori INFN-CNAF Viale Berti Pichat 6/2 40127 Bologna Italy Tel. : +39-051-6092920 Skype: andrea-cnaf
John, The only contentious difference between what you are arguing for and what I am arguing for is that in your case you require the grid to perform the interpolation. I argue that the grid, as one relying party, is _not_ the best place for this, as URs may go anywhere. If a document bearing only the state of the system at an instant in time does not go into the system with the logic to interpret that data (i.e. the algorithm and the sampling rates) then those documents are worthless. Therefore, I hold to my opinion that any interpolation must be done at the UR creation point (i.e. in the domain of the resource provider) regardless of whether their storage system has journalling, relies upon sampling or employs whatever alternative means to arrive at an integrated value. In this light I do not believe your argument has any weight stating that it is too difficult to supply an integral value when, at a bear minimum, the resource provider can perform locally the same algorithm that you would have the accounting system perform remotely. Further: Assuming sampling, the storage system has better indicators of how its facility is used. It will therefore be in a better place to increase sampling rates and adjust the algorithm used to achieve more accurate usage values should it need to do so.
Mike, I think you are way out of line here. As long ago as Munich OGF we discussed time integrals for UR and I thought we reached a consensus (well maybe everyone except you) that we could not ask storage systems to start evaluating integrals of storage usage for the use of UR. They don't do it and we would be asking them to implement journaling or to recalculate usage at every file interaction (or at least write/modify/delete). Just not on.
I do not believe I am "way out of line". I am not the only person to have the view that integrated values are a requirement of a definition of usage. The following people were at the Munich meeting: Gordon, John - STFC Kennedy, John - Rechenzentrum Garching (RZG) Keskrand, Kalle - EENet Kretzschmar, Michael - MNM-Team Maier, Andreas - LRZ Garching Raitviir, Tõnu - EENet Reetz, Johannes - Max-Planck-Institut für Plasmaphysik URBAH, Etienne - LAL, Univ Paris-Sud, IN2P3/CNRS Wolfrat, Jules - SARA I believe Etienne also shares my view strongly about integrals vs instantaneous values. I was not able to attend UR-WG in Munich as it clashed with security sessions. I am sorry for this. I believed you had a reasonable understanding of what would constitute a storage usage record. At that time I was happy to leave it at that.
I thought we all understood that this is a compromise for the pragmatic purpose of getting a UR soon that can be implemented in the real world.
I cannot find minutes of that meeting only the preprepared slides. The slides do not (in fairness as they were probably written prior to discussion cannot) support that statement of compromise.
You seem to enjoy the tautological discussion of what accounting should be in the theoretical world. I just want to start writing records tomorrow, if not today.
Please leave the rhetorical aspersions out. My "Tautological discussion" as you put it is not so. I repeat myself merely to put across the same argument in different words so as to help you and the group understand where I'm coming from. -- I have not yet heard a counter argument strong enough to make me put-up and shut-up.
In the longer term maybe we make requirements against storage systems that they implement integrals as well as user level and i/o accounting.
For now can we concentrate on agreeing something that can be implemented.
Any of this can be implemented and easily. And as I understand UR v1 should be sufficient for your needs as it stands. Mike
Hi John, I understand why you want the allocated resources and personally have nothing against us having an optional field for it. <ur:StorageResourceCapacityUsed>14728</ur:StorageResourceCapacityUsed> <ur:StorageResourceCapacityAllocated>50000</ur:StorageResourceCapacityAllocated> and I agree that this should be in the same record. I do however feel that a UR should cover a period of time and not just one point in time. I would argue that it is the job of the sensor or even better the storage system to be able to provide this information. The UR is there primarily to describe this information which can then be used for accounting/billing. We talked briefly in the last tele-con about a possible mis-usage of the record for a time duration of "0" which would then be a point in time record. I am starting to think we may need to actually tag the record as Point-in-Time (PiT) or Time-Integral(TI), but that's a secondary thought. The main point is that we could possibly cover both with the same record format... (or we may need two URs.). To be honest i need to think about it some more. The integral approach just seems more natural if we see accounting/billing as the end goal. * It fits better with the Compute UR (point-in-time doesn't fit at all). * You shouldn't need to do any mathematical gymnastics after the fact (no need to consider policies for min/max/avg usage) etc. * Missing records is something that you may have in Compute URs as well. I agree that in compute you are in a better situation since the missing records can often (if not always) be generated later since the batch system allow this - the storage systems are just weak here. It should be the job of the sensor software or storage system to be able to plug any gaps. That's what you do with Compute. Anyway, I think it's best to have your use-case written and then we can all discuss a bit more - prob best during the tele-con. cheers johnk On 02/21/2012 03:07 PM, john.gordon@stfc.ac.uk wrote:
Mike, you keep coming back to time integration. I think that once we decided on sampling rather than evaluating integrals then all we have is a series of observations and there are a variety of conclusions we can draw from them. None of them contain the word ‘always’ since we can never know what happens between observations.
Once one has a collection of records then different grids, projects, infrastructures may apply different procedures. I am not against the durationtime/elapsedtime argument but I won’t be attempting to construct time integrals from it. I will not be treating periods of time not covered by records as unused or empty space. I will be calculating min/max/mean over a time period with perhaps some weighting if the end-user projects or sites feel that this isn’t the full story. I might come back and propose a summary UR for passing this information around. I think that for many people desperate for storage accounting these simple metrics will be enough. If they are not then we can reconvene and change/extend the UR.
John
*From:*Mike Jones via mobile [mailto:mike.jones@manchester.ac.uk] *Sent:* 21 February 2012 10:56 *To:* Gordon, John (STFC,RAL,ESC); andrea.cristofori@cnaf.infn.it; j.k.nilsen@fys.uio.no *Cc:* ur-wg@ogf.org *Subject:* Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
To follow up on John's WLCG usecase: I agree that VO space allocated over a period of time _is_ a usage. BUT VO allocation at any instant is not. That said, the dynamics of any allocation/usage needs to be recorded to be able to answer the question "over period X did the allocation always satisfy: instantaneous_storage( used+available) > N (dimension:bytes)?" John, does that last clause satisfy WLCG?
Mike -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
john.gordon@stfc.ac.uk wrote:
I’ll be in Taiwan next week UTC+8 so I may or may not attend. Depends if anything official is happening that evening and if I can stay awake and if Skype works.
I’ll add the WLCG storage use case to the wiki before then. I must stress that this is important to WLCG and that we consider space allocated to a VO as ‘usage’ as it is not available to others and is to be recorded just like space occupied by files. i.e. Not monitoring any more than recording the existence of files in monitoring.
John
*From:*ur-wg-bounces@ogf.org [mailto:ur-wg-bounces@ogf.org] *On Behalf Of *Andrea Cristofori *Sent:* 21 February 2012 08:15 *To:* Jon Kerr Nilsen *Cc:* ur-wg@ogf.org *Subject:* Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET
Dear all,
Sorry for the short notice but neither I or Jon can make it to today phone conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF:
http://www.ogf.org/gf/event_schedule/index.php?id=2413
Again apologies.
Cheers, Andrea
On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote:
Hi all,
As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to
Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608
cheers,
Jon
On 7. feb. 2012, at 17:32, Andrea Cristofori wrote:
Dear all,
Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:
- AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)
Especially for John Gordon: can you please comment on those two use cases:
1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have.
All: seems to be more monitoring Comments from John are required
1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB).
All: seems to be more monitoring Comments from John are required
Cheers Andrea
-- ur-wg mailing list ur-wg@ogf.org <mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg
-- ur-wg mailing list ur-wg@ogf.org <mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg
-- Andrea Cristofori INFN-CNAF Viale Berti Pichat 6/2 40127 Bologna Italy Tel. : +39-051-6092920 Skype: andrea-cnaf
-- Scanned by iCritical.
-- ur-wg mailing list ur-wg@ogf.org https://www.ogf.org/mailman/listinfo/ur-wg
-- +------------------------------------------------------------+ |Dr. John Alan Kennedy Rechenzentrum Garching (RZG) | |Mail: jkennedy@rzg.mpg.de Boltzmannstrasse 2 | |Phone: +49 89 3299 2694 85748 Garching | |Fax: +49 89 3299 1301 | +------------------------------------------------------------+
John, thanks for your support. Your proposal is fine with me. I am happy to include a time period and my preference is for the time period since the last published measurement. One could then treat the measurement as an average over that time period. One could them combine records over a longer time period and rewrite the time period in a new record. I don’t think we can ask storage systems to deliver integrals. This has to be capable of use for a simple filesystem. John From: john alan kennedy [mailto:jkennedy@rzg.mpg.de] Sent: 22 February 2012 09:35 To: Gordon, John (STFC,RAL,ESC) Cc: mike.jones@manchester.ac.uk; andrea.cristofori@cnaf.infn.it; j.k.nilsen@fys.uio.no; ur-wg@ogf.org Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET Hi John, I understand why you want the allocated resources and personally have nothing against us having an optional field for it. <ur:StorageResourceCapacityUsed>14728</ur:StorageResourceCapacityUsed> <ur:StorageResourceCapacityAllocated>50000</ur:StorageResourceCapacityAllocated> and I agree that this should be in the same record. I do however feel that a UR should cover a period of time and not just one point in time. I would argue that it is the job of the sensor or even better the storage system to be able to provide this information. The UR is there primarily to describe this information which can then be used for accounting/billing. We talked briefly in the last tele-con about a possible mis-usage of the record for a time duration of "0" which would then be a point in time record. I am starting to think we may need to actually tag the record as Point-in-Time (PiT) or Time-Integral(TI), but that's a secondary thought. The main point is that we could possibly cover both with the same record format... (or we may need two URs.). To be honest i need to think about it some more. The integral approach just seems more natural if we see accounting/billing as the end goal. * It fits better with the Compute UR (point-in-time doesn't fit at all). * You shouldn't need to do any mathematical gymnastics after the fact (no need to consider policies for min/max/avg usage) etc. * Missing records is something that you may have in Compute URs as well. I agree that in compute you are in a better situation since the missing records can often (if not always) be generated later since the batch system allow this - the storage systems are just weak here. It should be the job of the sensor software or storage system to be able to plug any gaps. That's what you do with Compute. Anyway, I think it's best to have your use-case written and then we can all discuss a bit more - prob best during the tele-con. cheers johnk On 02/21/2012 03:07 PM, john.gordon@stfc.ac.uk<mailto:john.gordon@stfc.ac.uk> wrote: Mike, you keep coming back to time integration. I think that once we decided on sampling rather than evaluating integrals then all we have is a series of observations and there are a variety of conclusions we can draw from them. None of them contain the word ‘always’ since we can never know what happens between observations. Once one has a collection of records then different grids, projects, infrastructures may apply different procedures. I am not against the durationtime/elapsedtime argument but I won’t be attempting to construct time integrals from it. I will not be treating periods of time not covered by records as unused or empty space. I will be calculating min/max/mean over a time period with perhaps some weighting if the end-user projects or sites feel that this isn’t the full story. I might come back and propose a summary UR for passing this information around. I think that for many people desperate for storage accounting these simple metrics will be enough. If they are not then we can reconvene and change/extend the UR. John From: Mike Jones via mobile [mailto:mike.jones@manchester.ac.uk] Sent: 21 February 2012 10:56 To: Gordon, John (STFC,RAL,ESC); andrea.cristofori@cnaf.infn.it<mailto:andrea.cristofori@cnaf.infn.it>; j.k.nilsen@fys.uio.no<mailto:j.k.nilsen@fys.uio.no> Cc: ur-wg@ogf.org<mailto:ur-wg@ogf.org> Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET To follow up on John's WLCG usecase: I agree that VO space allocated over a period of time _is_ a usage. BUT VO allocation at any instant is not. That said, the dynamics of any allocation/usage needs to be recorded to be able to answer the question "over period X did the allocation always satisfy: instantaneous_storage( used+available) > N (dimension:bytes)?" John, does that last clause satisfy WLCG? Mike -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. john.gordon@stfc.ac.uk<mailto:john.gordon@stfc.ac.uk> wrote: I’ll be in Taiwan next week UTC+8 so I may or may not attend. Depends if anything official is happening that evening and if I can stay awake and if Skype works. I’ll add the WLCG storage use case to the wiki before then. I must stress that this is important to WLCG and that we consider space allocated to a VO as ‘usage’ as it is not available to others and is to be recorded just like space occupied by files. i.e. Not monitoring any more than recording the existence of files in monitoring. John From: ur-wg-bounces@ogf.org<mailto:ur-wg-bounces@ogf.org> [mailto:ur-wg-bounces@ogf.org] On Behalf Of Andrea Cristofori Sent: 21 February 2012 08:15 To: Jon Kerr Nilsen Cc: ur-wg@ogf.org<mailto:ur-wg@ogf.org> Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET Dear all, Sorry for the short notice but neither I or Jon can make it to today phone conference. We would like then to postpone it to next week (same day, same time). About today actions Jon already published the document and I got an answer for the slot at OGF: http://www.ogf.org/gf/event_schedule/index.php?id=2413 Again apologies. Cheers, Andrea On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote: Hi all, As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to Place: Skype Skype contact: lund.room.1 Phone bridge: +46462884608 cheers, Jon On 7. feb. 2012, at 17:32, Andrea Cristofori wrote: Dear all, Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following: - AC: Ask for another session the same day at OGF and update the agenda on OGF website - JKN: make public the document - All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache) Especially for John Gordon: can you please comment on those two use cases: 1. As a project I would like to be able to view the used and unused storage space that I have on a storage resource. - Thus I can see how much headroom I have. All: seems to be more monitoring Comments from John are required 1. As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource. - Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB). All: seems to be more monitoring Comments from John are required Cheers Andrea -- ur-wg mailing list ur-wg@ogf.org<mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg -- ur-wg mailing list ur-wg@ogf.org<mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg -- Andrea Cristofori INFN-CNAF Viale Berti Pichat 6/2 40127 Bologna Italy Tel. : +39-051-6092920 Skype: andrea-cnaf -- Scanned by iCritical. -- ur-wg mailing list ur-wg@ogf.org<mailto:ur-wg@ogf.org> https://www.ogf.org/mailman/listinfo/ur-wg -- +------------------------------------------------------------+ |Dr. John Alan Kennedy Rechenzentrum Garching (RZG) | |Mail: jkennedy@rzg.mpg.de<mailto:jkennedy@rzg.mpg.de> Boltzmannstrasse 2 | |Phone: +49 89 3299 2694 85748 Garching | |Fax: +49 89 3299 1301 | +------------------------------------------------------------+
participants (7)
-
Andrea Cristofori -
john alan kennedy -
john.gordon＠stfc.ac.uk -
Jon Kerr Nilsen -
Mike Jones -
Mike Jones via mobile
-
Ralph Mueller-Pfefferkorn