SAGA-RG design team meeting 12/13/2004 DK and SJ are taking minutes to post on the web CL- We?re just making a straw man document, not final API TG- minor timing details. Start 9AM CST each day; go 3 hrs Hartmut- sick, will be here weds on. SJ- everybody email shantenu TG- style details. OO vs. C and fortran. Discussion- target audience, how to make API which is OO for C and fortran. Scripting interfaces vs. programming API Chris Smith: notification? DK- low priority: asynch CL- priority list. Hrabri- similar issues in drmma. Some ops return right away. For synch ops, use timeouts. Discussion around style/design of polling/async, callbacks, etc. AM- Java CoG- simple API, synchronous. On top ? task module. TG- start w/ nonblocking calls & build blocking calls around it CL- how much oo? No Multiple inheritance, JS- c-style OO. Common ops across handles Extensive discussion re: handle objects, which can point to different types of files TG- How to support multiple backend file types? SJ- how much of OO are we going to use- no simple inheritance? DK- structured error handling. Error handle which can be queried to get all exception Information TG- error tree (since a single call can have ?multiple errors? from different tplugins) TG- style. Security. CL?security mech as base part of api? TG- cmplex if multiple security tokens have to be available CL- features sppted by api will determine security features exposed CS- frequent mode- get_auth_token(), provide token to api calls TG- session handling must support more than one auth type CL- do we have a style/approach to security? Consensus: ?session handle? JS ? explicit management of security via API, or implicitly (user runs grid-proxy-init in shell, then runs program) TG- next step of agenda- use cases First: Cover the ?Related groups in GGF? document by Andre and Gab TG- resource management vs. job startup AM- document should have been more fine-grained? Subcategorization of job reosources (job submission, job control, job monitoring, resource discovery, resource description) What do we mean by data access? Files Databases Remote file access Data management?treat file as object to move around Databases?information management/lookup. Remote access to DBs? Genome sequences. Life science consortia working on this? DAIS- unique interface to dbs. Exposed interface is not that complex Extensive discussion of meaning of ?data X? Gregor- want to narrow down & focus JS- data streaming underlying upper level features. Current apis to do authenticated sockets are crappy SJ- logical files Logical file (such as replica catalog) CS- does steering mean checkpointing? Sept category? Identifying ?objects? we will be dealing with- Files Jobs Databases Applications Stream Logical files Event Advertisement(?) Should visualization be *specifically* implemented in the API? No. App itself does all that; just uses API for things like streaming data Information- publish & search for info Communication- Messaging Information encoding/organization (?) Data access/streaming/events?flavors of communication Bulk operations?one per each distinct section (job, file, etc) Errors, Audits, transactions, workflows: Error ?objects? Audit ?objects? Log function in api? Extensive discussion on meaning of auditing and whether it belongs in the api Maybe it should be low priority? Still a standing issue Enabling/disabling logging desirable CS- Is there standard audit record format?and can we put hooks into api? Transactions? Probably not Workflow? CS?workflow, bulk, task ? how related? Task = interface to define task, asynch run, notification callbacks Task-> interfaces unique to type of operation One job at a time; synch and asynch task Cog kit- task specification, arb. Task, bind to resource Cog kit- Event model. Being adopted by GT 4. Task- when event changes in task, fire something back/throw exception Workflow- built on top of task. Doesn?t need its own API. Just dependent tasks QoS?lightpaths, LSPs, others (JS) and diffserve. Application Interfaces- ?I want this much bandwidth?. Do we want an API for this set of actions? CL?leave it till later, but make sure we don?t design ourselves into a corner so we can?t support it Priorities/TODO: Files/Data streaming Jobs/Tasks Simple hooks for resource discovery Error handling Set aside: Events Workflow QoS Auditing Information advertising, search Steering Considerations Bulk operations Security Eliminate: Transaction databases Agenda for next session: What objects in different areas? After lunch Job APIs DRMAA JSDL (language) Globus (GRAM) Ssh Unicore (abstract job object) Condor SOAP GAT CoGKits (python/java) Gregor walked us through the CoG Task/Job interface (globus-cog-api.pdf and javacog.pdf) TG- Other APIs: Extensive in-depth discussions about task/job interfaces, use cases. CS- Platform?s ?lsrun? CS walked us through the Platform LSF SDK Discussion of what people are going to do overnight Jobs Files Posix-like I/O Dec 14 2004 Jobs: CS read over CoG, GAT API. Job service object; job description option Wants more direct mapping between job objects and JSDL Discussion of relationship between AM- what about DRMAA? CS- very similar to our C api What does job id mean across APIs? In ssh? Job id?go to portal and reference your running Data streams: Need authentication, authorization; not so much call for explicit encryption Opaque security token IO handle: read, write, disconnect Server side- must be able to connect/disconnect, without disturbing simulation Don?t expose fine-grained stuff (socket options) Iostream = open(sc, sd) Send(iostream) Receive(iostream) Disconnect(iostream) Service_open(sd, auth, enc, authorize), read, write, disconnect Service description opaque in case impl has to deal with firewall, rather than just host and port Associate ACLs with the service connection Files: AM- presented file and directory interfaces Discussion around iterator vs. full list of files in directory calls Discussion re: should ?directory object? have state like current directory; does it belong in the service object? Directory- cp, mv, rm (within and across namespace) etc Files- read, write, seek, etc Extended read/write calls for efficiency Above is physical file/directory; extend this to logical files/directoryes New directory methods: add_location, del_location, get_locations(), replicate() (IE for a replica service) Locking?should we have operations? Jobs: where should the general property list for job descriptions come from CS- JSDL Extensive discussion of what JSDL is, what it isn?t, and how it maps to SAGA Extensive discussion & hashing out of the job API Extensive discussion &hashing out of the secure channel API Weds December 15 2004 Discussing file interface. Keep POSIX semantics AMAP. Discussion of various readdir, getdents, other calls Long discussion on task interface, eventually stopped and Went back to files Discussion of async vs. sync routines, what are appropriate things to expose in things like stat() return value?