March 2015 - glue-wg - lists.ogf.org

When is data stale?
by Paul Millar 07 May '15

07 May '15

Hi all, I had a recent discussion with a site over the meaning of Entity.CreationTime and Entity.Validity. I wanted to share my thoughts and see whether others have the same view and whether this might be worth documenting: either as part of an updated GLUE document or as some auxiliary document (a "GLUE Processing Model" perhaps). The core question is this: how to know when published data is stale? Prima facie, this is easy: simply add Entity.Validity to Entity.CreationTime. If the resulting time is in the past then the object is stale. It becomes slightly tricky when considering an object that passes though several caching agents, with each agent pulling updated information periodically. [For those that don't know, this is how the EGEE information system currently works; there are three caching levels ("resource-level", "site-level", "top-level"). Each level has the same software, "BDII", with different configuration.] First off, I assert that the current info-provider model (a script that provides the current up-to-date information) cannot publish Entity.Validity. With a pull-update model, the info-provider cannot know when the next request will come. (To illustrate this, consider two agents calling the info-provider with different schedules, or an info-provider that supplies information "on demand" whenever a user clicks on some web-page). As the info-provider cannot know when the next request will come, it cannot set an Entity.Validity value. However, the info-provider can publish an Entity.CreationTime. Let's suppose it does. Let's also suppose that the first agent (the resource-level BDII) queries every two minutes. Ideally, the BDII will read the data and calculates what Entity.Validity is needed so that Entity.Validity+Entity.CreationTime is two minutes in the future. It then records this Entity.Validity along with the original Entity.CreationTime. If (somehow) this data is observed after the two minutes has elapsed, it is clear that the data is stale. Note that it is also possible (i.e., logically self-consistent) for the BDII to reset the Entity.CreationTime to the current time. I suggest this isn't done as knowing the original creation time could be quite useful. The same procedure would happen for the higher levels: each one would calculate a (potentially new) Entity.Validity so that the object will not be valid after it anticipates fetching fresh data. This calculated Entity.Validity would replace any that already exists in the supplied data. Assuming you agree with this approach, there are some open questions: 1. should the agent (BDII) reset Entity.CreationTime (I'd say "no"), 2. should the agent (BDII) add an Entity.CreationTime if the source does not provide one? 3. what should the agent (BDII) do when faced with stale data? Should it simply log a warning or should it reject the data? Any thoughts? Cheers, Paul.

7 23

GLUE WG teleconference, Tuesday, March 10, 2015
by JP Navarro 25 Mar '15

25 Mar '15

Greetings GLUE WG, Our next GLUE WG teleconference will be: : Tuesday, March 10, 2015 : 15:30-17:00 Central Europe, 9:30-11:00 AM US Central NOTE: Due to the US Daylight Savings time change this weekend, this meeting will be at the usual time in Europe BUT one hour later than usual in the US. We will try this teleconference and screen sharing tool: : Web: https://meet.illinois.edu/herrio/V57YV10Q : --> Cancel login dialog, click guest "Sign in here instead", enter name, click "Join the meeting" : Phone: 1-888-983-3631, International 1-217-332-6338 or 1-312-994-8410 : Conference ID: 6759150 As a fall back we also have: : Web: https://webconf.vc.dfn.de/ogf-glue-wg/ (Adobe Connect) : Login as Guest using your real name Meeting materials folder: : http://redmine.ogf.org/dmsf/glue-wg?folder_id=6588 Proposed agenda with previous meeting notes: 1) LDAP rendering: public comment is closed, discuss next steps Previous: > Action: Florido will send LDAP schema to merge into document, recommend publish path > Action: JP/Shiraz will merge Florido's 9/2/2014 comments, LDAP document and release for public comment > Agreed that we would publish the LDAP compiled schema at a specific URL > Action: JP to confirm schemas.ogf.org versus www.ogf.org/schemas/ (prefer schemas.ogf.org) > CONFIRMED that we can use schema.ogf.org/ to publish LDAP and JSON schema > We do not think there will be technical conflicts for XML, LDAP, and JSON schemas co-existing under the same URL namespace. > http://www.ogf.org/documents/GFD.84.pdf defines standard XML namespace. OGF doesn't have equivalent standards for LDAP or JSON. > Schema will be published as "*.schema.ldap" > Shiraz/JP review our main wiki page and make sure there are links to all the published schemas. 2) Enumerations: approve enumerations process document Previous: > Florido posted script to merge, it's on Github. > David volunteers to help manage enumerations, including add missing enumerations. > We need define the enumerations described in GFD.147. > ACTION: Warren will post proposed job state enumerations from XSEDE > ACTION: David/Warren/Shiraz will explore Git tools/ pull requests for maintaining enumerations and propose changed enumeration management process documentation 3) Cloud Extensions: Do we have champions to move forward with GLUE 2.1? Champions: Francisco Brasileiro and Abmar Barros from Federal University of Campina Grande - www.ufcg.edu.br Peter Solagna and anyone else from EGI? Previous: > Our next meeting on November 25 will focus on GLUE 2.1 Cloud changes > We discussed extensively what we mean by compatibility between GLUE 2.0 and 2.1. > No conclusion reached and needs to be further discussed. > We didn't have time to discuss Paul's proposal to add a View attribute in 2.1 > David will explore pricing policies in the context of GLUE2 and OGF SLA (WS-Agreement, GRAP) working group > Terminology: Call this GLUE 2.1 specification, 100% compatible with 2.0 with additional optional cloud extensions 4) JSON: continue rendering discussions, discuss next steps. Previous: > JSON schema is ready for WG review before release for public comment. > ACTION: Warren will post schema pointer to the working group this week, we will vote on 25th to release for public comment. > Moved to release for public comment after correcting schema reference URL. > XSEDE is implementing JSON GLUE2 > David funding supports his work on XML and JSON > UNICORE implementing REST services and may implement JSON if their is a need > Shiraz/JP will review wiki page for tools and/or deployments 5) Future meetings Should we meet at OGF44 co-located with the EGI Conference 2015 May 18-22, 2015, in Lisbon, Portugal? Regards, JP and Shiraz

5 6

JSON schema submitted for public comment
by JP Navarro 10 Mar '15

10 Mar '15

From today's meeting: > 4) JSON: continue rendering discussions, discuss next steps. > > AGREED: Release it for public comment. > Action (JP/Shiraz): Submit it for public comment. Submitted: https://redmine.ogf.org/issues/261 JP

1 0

Enumerations from ARC/NorduGrid required to be on git-hub
by Florido Paganelli 06 Mar '15

06 Mar '15

Hi all, I got a ticket for a couple of enumerations belonging to ARC that were evaluated as non-existent. As some of you know the GLUE2 validator ran by EGI and developed by the BDII team couldn't benefit of the enumerations on git-hub since that listing is not complete. (thanks JP for the minutes btw :D ) This is now bouncing back to the software I maintain because the validator complains that the InterfaceName strings my collaboration added four years ago are not known. So they are, because they're not in GFD.147 and they never made it to git-hub. Having to put my hands on that again, I'll try to contribute a bit better. Some of you might remember that I had a long list of clashing open enumerations (i.e. multiple enumerations that identify the same software/component/interface) that we never sorted out. Since I need to fix this for my collaboration I can find and approach to sort out the others as well. Regarding how we do this, I seemed to remember David wanted to propose a protocol similar to the one I wrote but more based on pull requests on git-hub instead of writing to the group. We can try to implement it for this round if he feels like, that is, I will put down the list of stuff I had and try to merge it with what is it on git-hub, and perform a git pull request to the group repo. Let's see if it makes it easier to perform the procedure. David do you have any sugestion how I should proceed otherwise? Regarding the clashing names, I will take them aside from the pull request and see if I can sort it out some other way. Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Location: Fysikum, Hus B, Rum B313 Office Tel: 046-2220272 Email: florido.paganelli(a)REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================

1 0

GLUE WG teleconference, Tuesday, December 2, 2014
by Navarro, John-Paul F. 03 Mar '15

03 Mar '15

Greetings GLUE WG, Our next GLUE WG teleconference will be: : Tuesday, December 2, 2014 : 15:30-17:00 Central Europe, 8:30-10:00 AM US Central NOTE: We had previously discussed meeting last week on November 25th, but that didn't work out. Hoping this week works out for most. We will try this teleconference and screen sharing tool: : Web: https://meet.illinois.edu/herrio/V57YV10Q : --> Cancel login dialog, click guest "Sign in here instead", enter name, click "Join the meeting" : Phone: 1-888-983-3631, International 1-217-332-6338 or 1-312-994-8410 : Conference ID: 6759150 As a fall back we also have: : Web: https://webconf.vc.dfn.de/ogf-glue-wg/ (Adobe Connect) : Login as Guest using your real name Meeting materials folder: : http://redmine.ogf.org/dmsf/glue-wg?folder_id=6588 Proposed agenda: Brief updates followed by a more detailed discussion of proposed cloud extensions. 1) LDAP rendering: assess final wg comments and votes and perhaps forward for public comment Previous: > Action: Florido will send LDAP schema to merge into document, recommend publish path > Action: JP/Shiraz will merge Florido's 9/2/2014 comments, LDAP document and release for public comment > Agreed that we would publish the LDAP compiled schema at a specific URL > Action: JP to confirm schemas.ogf.org versus www.ogf.org/schemas/ (prefer schemas.ogf.org) > CONFIRMED that we can use schema.ogf.org/ to publish LDAP and JSON schema > We do not think there will be technical conflicts for XML, LDAP, and JSON schemas co-existing under the same URL namespace. > http://www.ogf.org/documents/GFD.84.pdf defines standard XML namespace. OGF doesn't have equivalent standards for LDAP or JSON. 2) Enumerations: approve enumerations process document Previous: > Florido posted script to merge, it's on Github. > David volunteers to help manage enumerations, including add missing enumerations. > We need define the enumerations described in GFD.147. > ACTION: Warren will post proposed job state enumerations from XSEDE > ACTION: David/Warren/Shiraz will explore Git tools/ pull requests for maintaining enumerations and propose changed enumeration management process documentation 3) Cloud Extensions: compare benefits and disadvantages of both approaches Previous: > Our next meeting on November 25 will focus on GLUE 2.1 Cloud changes > We discussed extensively what we mean by compatibility between GLUE 2.0 and 2.1. > No conclusion reached and needs to be further discussed. > We didn't have time to discuss Paul's proposal to add a View attribute in 2.1 > David will explore pricing policies in the context of GLUE2 and OGF SLA (WS-Agreement, GRAP) working group 4) JSON: continue rendering discussions, discuss next steps. Previous: > JSON schema is ready for WG review before release for public comment. > ACTION: Warren will post schema pointer to the working group this week, we will vote on 25th to release for public comment. 5) Future meetings Discuss December 12th meeting in London. Regards, JP and Shiraz

4 3