- ogsa-wg - lists.ogf.org

RE: [cddlm] Basic POSIX Component & template draft
by Dave Berry 08 May '06

08 May '06

Hi, It sounds like you're having interesting discussions. What concerns me is how does this work relate to: - the hierarchical naming level of the OGSA naming system? - the work of the Grid File System working group? - ByteIO? - the OGSA Data Architecture? Dave. > -----Original Message----- > From: owner-cddlm-wg(a)ggf.org [mailto:owner-cddlm-wg@ggf.org] > On Behalf Of Hiro Kishimoto > Sent: 08 May 2006 14:56 > To: Steve Loughran > Cc: OGSA WG; cddlm-wg(a)ggf.org > Subject: Re: [cddlm] Basic POSIX Component & template draft > > Hi Steve, > > Your file & filesystem components are very good. They are > what I am looking for. One more question I have in my mind is > how to set user-id, group-id, and permissions to deployed files. > > Can you call in OGSA-EMS session on Wednesday? > It starts 5:45pm JST = 9:45am UK. > > The dial-in number for this session; > US: +1 718 3541071 (New York) or > +1 408 9616509 (San Jose) > UK: +44 (0)207 3655269 (London) > Japan: +81 (0)3 3570 8225 (Tokyo) > PIN: 4371991 > > Thanks, > ---- > Hiro Kishimoto > > Steve Loughran wrote: > > Hiro Kishimoto wrote: > >> Hi all, > >> > >> Jun Tatemura and I took a home work assignment about "Basic POSIX > >> Component" for BLAST application deployment. > >> > >> Attached is my first-cut draft and hope to discuss at > GGF17 OGSA EMS > >> session. > >> Please have a look and give your feedbacks before or at the GGF17. > >> Your feedbacks are very important. > >> > >> Thanks and see you in Tokyo! > > > > > > This is really good, and I do agree, we do need a set of > foundational > > things to deploy. > > > > One thing I would argue, having learned the lessons of both Ant and > > SmartFrog, is that a consistent model of file system > components comes > > first. That is, we need components to model > > > > -files (with a liveness test that verifies the file is > present) -paths > > (an ordered list of files/directories) -directories (with > components > > that can create a dir on deployment, optionally to delete > it and its > > contents on termination) -wildcarded sets of files (**/*.csv) > > -temporary files and directories -cached downloads -text files > > > > I'm attaching the PDF file describing the current smartfrog set of > > components that do this. The underlying way that this is > done is that > > all components that consider themselves to be sources of > > filenames/paths set the attribute absolutePath at runtime. They > > usually also export an RMI interface for RPC-style > operations, along side their RESTy state. > > > > A standard set of filesystem components lets you include the > > operations to create and clean up directories, temporary files and > > other housekeeping operations into the workflows of a deployment. > > > > Looking at the Posix proposal, I'd suggest starting with > the reusable > > set of filesystem types, including a <fs:mount> component > that could > > mount a local or remote filesystem, and which would then pass the > > local location to interested parties as the > filesystem:absolutePath attribute. > > > > You absolutely don't need to introduce a new shared datatype > > FileSystemName, because that implies some kind of alternate > > naming/locating scheme outside of what CDL and the runtime > has built in. > > Dynamic CDL references are sufficient to crosslink > information inside > > the graph of deployed things, and offer flexibility as well > as ease of > > management (you see them when you walk the graph). > > > > First, we'd need filesystem types: > > > > <tmp cdl:extends="fs:TempFileSystem"> > > <fs:Description> ... </fs:Description> > > <fs:DiskSpace> > > <fs:LowerBoundedRange>10737418240.0</fs:LowerBoundedRange> > > </fs:DiskSpace> > > </tmp> > > > > <home cdl:extends="fs:Directory"> > > <fs:Description>Chris's home directory</fs:Description> > > <fs:dir>/home/csmith</fs:dir> > > </home> > > > > When deployed, both of these would add absolutePath as an > attribute, > > so that tmp/absolutePath woudl resolve to, say, tmp/work01234/ and > > home:absolutePath to /home/csmith. > > > > > > You'd then mount a blast dir, say on an nfs filesys whose URL you > > provide > > > > <blastfs cdl:extends="fs:RemoteFileSystem"> > > <fs:url>nfs://filestore/csmith/blast</fs:url> > > <fs:mountUser>csmith</fs:mountUser> > > </tmp> > > > > The database file is under here, and you declare that it > must exist. > > Deployment will fail > > if it is absent: > > > > <db cdl:extends="fs:File"> > > <fs:dir cdl:ref="/tmp/absolutePath" cdl:lazy="true" /> > > <fs:filename>db/ncbiblast/est</fs:filename> > > <fs:filemustexist>true</fs:filemustexist> > > </db> > > > > > > To use it, just refer to its path > > > > <blast:database cdl:ref="/db/absolutePath" cdl:lazy="true"/> > > > > Paths could be represented by some list-like element, whose entries > > are evaluated at deploy time and then a platform-specific > path created > > as a result > > > > <path cdl:extends="fs:Path"> > > <pathentry cdl:ref="/blastfs/absolutePath" cdl:lazy="true" /> > > <pathentry cdl:ref="/home/absolutePath" cdl:lazy="true"/> </path> > > > > On Windows /path/absolutePath would become something like > > "\\filestore\csmith\blast;c:\documents and settings\csmith" > > while on unix it could be > > "/nfs/filestore/csmith/blast:/home/csmith" > > > > This information now goes down to the jsdl component > > > > <jsdl-posix:Environment name="PATH" > > cdl:ref="path:absolutePath" cdl:lazy="true"/> > > <jsdl-posix:Environment name="TMPDIR" > > cdl:ref="tmp/absolutePath" cdl:lazy="true"/> > > > > > > -Steve > > > > > > > > > > > > > > > >

2 1

Basic POSIX Component & template draft
by Hiro Kishimoto 08 May '06

08 May '06

Hi all, Jun Tatemura and I took a home work assignment about "Basic POSIX Component" for BLAST application deployment. Attached is my first-cut draft and hope to discuss at GGF17 OGSA EMS session. Please have a look and give your feedbacks before or at the GGF17. Your feedbacks are very important. Thanks and see you in Tokyo! -- Hiro Kishimoto

2 3

RE: [ogsa-wg] Initial draft of HPC profile working group charter
by Dave Berry 07 May '06

07 May '06

Hi, All the discussion I've seen about this profile so far has focussed on specifying and submitting jobs, but the proposed charter says that this is only one of two areas of interest; the other being data staging. The immediate question is what link will this profile work have to the Data Movement Interface Specification group? Will the profile refer to the DMIS spec in the same way that it will refer to JSDL and BES? (Of course, JSDL also covers data staging to some extent). Work is also continuing on the data aspects of the OGSA architecture, in the OGSA Data WG. I would encourage the HPC profile group to engage with the OGSA Data group and the DMIS group concerning how to fit data staging into the overall architecture. Unfortunately I won't be in Tokyo next week but it would be useful if members of these groups could get together there to agree a working relationship. Best wishes, Dave. _____ From: owner-ogsa-wg(a)ggf.org [mailto:owner-ogsa-wg@ggf.org] On Behalf Of Marvin Theimer Sent: 05 May 2006 01:48 To: ogsa-wg(a)ggf.org Subject: [ogsa-wg] Initial draft of HPC profile working group charter Hi; Enclosed is an initial draft of the OGSA HPC profile working group charter. This will be brought up for discussion at the HPC profile BoF at the upcoming GGF17 conferenece in Tokyo. In the mean time, please feel free to comment on the draft via email to this mailing list. Thanks, Marvin.

3 3

Initial draft of HPC profile working group charter
by Marvin Theimer 07 May '06

07 May '06

Hi; Enclosed is an initial draft of the OGSA HPC profile working group charter. This will be brought up for discussion at the HPC profile BoF at the upcoming GGF17 conferenece in Tokyo. In the mean time, please feel free to comment on the draft via email to this mailing list. Thanks, Marvin.

2 2

Straw man design proposal for HPC profile
by Marvin Theimer 06 May '06

06 May '06

Hi; The virtual design effort around the HPC Profile initiative has created a use case document, has discussed extension mechanisms, and has received a straw man design proposal from Ian Foster and Dave Snelling. Chris Smith and I have put together a second straw man design proposal so that we have two different proposals to discuss at the HPC Profile BoF in Tokyo next week. That proposal is included as an attachment to this email. Whereas Ian and Dave's ESI design had as its primary focus a unification of the Globus and Unicore designs, the focus of Chris and my design proposal is an evolutionary design in which there is a base case and numerous optional extensions. Of course, it also represents our views on how we think the HPC profile should look. :-) The enclosed paper is intentionally an unfinished design since our goal is to engender discussion and obtain feedback from the grid community for the HPC Profile effort. So we invite comments and feedback, both in email to this mailing list and in person during the BoF next week in Tokyo. Thanks, Marvin (and Chris).

1 0

GGF17 information
by Hiro Kishimoto 05 May '06

05 May '06

Hi all, You can find some useful information for GGF17 at the following wiki site. Thanks Satoshi Matsuoka and Joel Replogle. http://testforge.ggf.org/sf/go/wiki1140 - General Directions from Narita airport to downtown Tokyo - Japan Cellular Phone Survival Guide for Foreign Visitors Hope it helps, -- Hiro Kishimoto

1 0

Resignation as Chair of OGSA-RSS
by Mathias Dalheimer 05 May '06

05 May '06

Hi all, during the preparation of the sessions for Tokyo this week, I noticed that I didn't contribute to OGSA-RSS in the last few months. Thinking about the reasons, I discovered two of them: I am sure this is true for all of us, but my time available for GGF is very limited. Due to other commitments in the last months, I wasn't able to spent more time on OGSA-RSS. The more important reason is that my work has changed in the last months. OGSA-RSS is going to define a scheduling interface which will be applicable to most grid schedulers. The interaction with an RSS- compliant consists of a simple request-response scheme, which is a sensible choice because of the interoperability such a scheme facilitates. My current work focus lies in an area where such a scheme is too simple. I think of a scheduler more in the sense of an agreement- finding and monitoring engine, which is a highly specialized view of the area. I think it is very hard to standardize more than the data formats, e.g. how to describe jobs, in this scenario. This is different from my starting point a year ago, when a mapping of my work to the OGSA-RSS seemed easy. I value the GGF as a place to meet people and exchange ideas, and I will continue to visit future GGFs for this. But I will resign as cochair of OGSA-RSS after GGF17. I hope you understand that I do not criticize the standardization efforts in GGF as a whole - in many other scenarios, the standardization has a lot of benefits. See you in Tokyo for good discussions, Mathias __________________________________________________________________ Dipl.-Wirtsch. Ing. (Inform.) Mathias Dalheimer Fraunhofer Institut fuer Techno- und Wirtschaftsmathematik (ITWM) Fraunhofer-Platz 1 67663 Kaiserslautern, Germany Phone +49(0)631/31600-4484 | mailto:dalheimer@itwm.fraunhofer.de Fax +49(0)631/31600-1099 | __________________________________________________________________

1 0

Version 6 of the information modeling guidelines
by Fred Maciel 05 May '06

05 May '06

Hi, I just uploaded a new version of the modeling guidelines document. I included most of the comments of the last F2F. I'm not showing in the document changes that just move text around. The latest version, as always, can be found at: https://forge.gridforum.org/projects/ogsa-wg/document/Modeling_Guideline s/en In the information model session in GGF17 the focus will be on Ellen's work, namely the position paper on the model for EMS (she is writing it as a separate document). We will only touch the guidelines document if there is plenty of time left in the session [please insert hysterical laughter here]. So if you have comments on the guidelines document, please grab me in the corridors in GGF17 or send an e-mail to the reflector. See you in Tokyo, Fred Maciel Hitachi America R&D The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking any action in Hitachi America, Ltd. on the contents of this information is strictly prohibited and is unlawful. If you have received this communication in error, please notify us immediately by responding to this e-mail and then delete it from your system. Hitachi America, Ltd. is not liable for any improper and/or incomplete transmission of the information contained in this communication or for any delay in its receipt. Hitachi America, Ltd. is not responsible for any damage caused by malicious code or alteration by any third party. The contents of this e-mail do not necessarily represent the views or policies of Hitachi America, Ltd.

1 0

Re: [saga-rg] Re: [ogsa-bes-wg] Extensible state model for jobs
by Christopher Smith 04 May '06

04 May '06

I agree ... we can discuss this next week. -- Chris On 03/5/06 10:09, "Thilo Kielmann" <kielmann(a)cs.vu.nl> wrote: > Chris and Andre, > > I think we really should resolve this issue face-to-face next week at GGF17. > That will save a lot of email typing time... > > Andre, do we have a slot in one of our sessions? > > > Thilo > > On Wed, May 03, 2006 at 06:08:46PM +0200, Andre Merzky wrote: >> X-Original-To: kielmann@localhost >> Delivered-To: kielmann(a)localhost.cs.vu.nl >> Delivered-To: grdfm-saga-rg-outgoing(a)mailbouncer.mcs.anl.gov >> X-Original-To: grdfm-saga-rg(a)mailbouncer.mcs.anl.gov >> Delivered-To: grdfm-saga-rg(a)mailbouncer.mcs.anl.gov >> Date: Wed, 3 May 2006 18:08:46 +0200 >> From: Andre Merzky <andre(a)merzky.net> >> To: Christopher Smith <csmith(a)platform.com> >> Cc: Andre Merzky <andre(a)merzky.net>, >> "ogsa-wg(a)ggf.org" <ogsa-wg(a)ggf.org>, ogsa-bes-wg(a)ggf.org, >> SAGA RG <saga-rg(a)ggf.org> >> Subject: [saga-rg] Re: [ogsa-bes-wg] Extensible state model for jobs >> >> Quoting [Christopher Smith] (May 03 2006): >>> >>> On 03/5/06 05:36, "Andre Merzky" <andre(a)merzky.net> wrote: >>> >>>> Right now its absent because only those states which can be >>>> actually reached by SAGA method calls are present. Now, as >>>> you said earlier, it can be assumed a flaw that SAGA has no >>>> means to put a job into 'Pending' state. >>>> >>> >>>> So, if there are some use cases or some agreement that the >>>> API should allow to move a job into 'Pending' state, the >>>> method/attribute will be added, and the state will be added. >>>> >>> The state that a job enters after a create_job operation is 'pending', so >>> the create_job operation is the method that puts a job in this state. >> >> Right, I understand that from the queuing point I think. >> However, from the application level it does not matter, as >> it cannot actively change that state. We have no notion of >> queue states at all. >> >> Well, let me put it differently: what do we gain by moving >> the state to the higher level? It complicates the state >> diagram (not much, that is not the point), but does not >> allow to perform any new state transition from application >> level. >> >> As said, I would compare this to Suspended state: its >> clearly an important state in many systems, but as long as >> the API does not expose means to suspend/resume a job, the >> state is meaningless, and merily informative (e.g. the job >> gut suspended by a 3rd party, or by the backend). That >> information is exposed however, through the job state >> details - same holds for hols (ha!) I think. >> >> >>> This state is required for queuing systems ... I can't even imagine how it >>> got dropped from the state diagram. I think you're trying to map the task >>> state model too closely. ;-) >> >> Nono, that has nothing to do with the task states - they >> could easily leave that state out. Well, its very >> convenient to have the same model here, thats for sure ;-) >> >> >>>> Similar reasoning holds for 'Suspend' - as long as we don't >>>> have suspend/resume methods, we would not like to expose a >>>> 'Suspended' state on the higher level state diagram. >>> >>> This I agree with. >> >> As said, it might well be that SAGA is broken in the respect >> that we doesn't allow job.hold (), or submit into hold. >> Well, it wasn't in the use-cases ... :-P >> >> Cheers, Andre. >> >> >>> -- Chris >> >> >> >> -- >> "So much time, so little to do..." -- Garfield >> > >

2 2

CFP GridNets 2006 : DEADLINE APPROACHING
by Antoine.Pichot＠alcatel.fr 04 May '06

04 May '06

Call for Papers =============== GridNets 2006 Third International Workshop on Networks for Grid Applications Co-Sponsored by IEEE Communications Society and Create-Net Co-located with IEEE Communications Society/Create-Net Broadnets 2006 San Jose, California, USA, October 1-2, 2006 http://www.gridnets.org Grid developers and practicioners are increasingly realising the importance of an efficient network support. Entire classes of applications would greatly benefit by a network-aware Grid middleware, able to effectively manage the network resource in terms of scheduling, access and use. Conversely, the peculiar requirements of Grid applications provide stimulating drivers for new challenging research towards the development of Grid-aware networks. Cooperation between Grid middleware and network infrastructure driven by a common control plane is a key factor to effectively empower the global Grid platform for the execution of network-intensive applications, requiring massive data transfers, very fast and low-latency connections, and stable and guaranteed transmission rates. Big e-science projects, as well as industrial and engineering applications for data analysis, image processing, multimedia, or visualisation just to name a few are awaiting an efficient Grid network support. They would be boosted by a global Grid platform enabling end-to-end dynamic bandwidth allocation, broadband and low-latency access, interdomain access control, and other network performance monitoring capabilities. The Gridnets 2006 workshop will provide a focused and highly interactive forum where researchers and technologists will have the opportunity to present and discuss leading research, developments, and future directions in the Grid networking area. As an IEEE publication, the proceedings will also be accessible through the IEEE Xplore website and other digital libraries. Best papers will be considered for publication in a special section of Elsevier Future Generation Computer Systems (FGCS) - The International Journal of Grid Computing: Theory, Methods and Application". Scope ------ The GridNets 2006 workshop will focus on research issues and challenges as well as lessons learned from experience. Topics of interest include and are not limited to: * New concepts and requirements to shape the design of eScience and Research Networks * Integration of advanced optical networking technologies and architectures (OPS, OBS) for the Grid environment * Coordination of network resources with other Grid resources (CPU, Storage) * Layer interactions: optical layer with higher layer protocols * Grid advanced resource reservation * Self-healing Grid networks * Traffic characteristics and performance analysis * New architectures and technologies that address Grid requirements * Experience on production-level optical network infrastructures * Middleware design and grid layer integration issues for accessing and managing network resources * Routing and scheduling for dynamic bandwidth control * Monitoring, provisioning, brokering of network resources * New multi-service frameworks and models * End-to-end application level control of network resource * Peer-to-peer approach for Grid networks * Network support for wireless Grids * Novel data transport protocols designed for new application services * Data replication and multicasting strategies and protocols * Fault-tolerance, protection, security, and scalability issues related to connecting large number of sites * Network cost, performance, and incentive issues * Grid network simulation * Identification of Grid peculiarities that network mechanisms can build upon Important Dates ----------------- Paper Submission Deadline: May 26, 2006 Paper Acceptance Notification: June 30, 2006 Final paper submission: July 31, 2006 (For submission instructions, refer to http://www.gridnets.org ) Organizing Committee ==================== Workshop co-chairs: Pascale Vicat-Blanc Primet (ENS Lyon), Michael Welzl (University of Innsbruck) Workshop vice-chair: Piero Spinnato (Create-Net) Technical Program Committee chair: Wayne Clark (Cisco) Panel chair: Bela Berde (Alcatel) Publicity chair: Antoine Pichot (Europe) (Alcatel), Tomohiro Kudoh (Asia) (AIST), Brian Tierny (USA) (LBL) Steering Committee: Imrich Chlamtac (Create-Net), Gigi Karmous-Edwards (MCNC), Michael Welzl (University of Innsbruck) Conference coordination and registration: Kitti H. Kovacs (ICST) Program Committee ================= Bill Allcock - Argonne National Lab Micah Beck - University of Tennessee Augusto Casaca - INESC Piero Castoldi - Scuola Superiore Sant'Anna Cees De Laat - Universiteit van Amsterdam Peter Dinda - Northwestern University Jose Fernandes - FCCN Tiziana Ferrari - INFN - CNAF Gabriele Garzoglio - Fermi National Lab Wolfgang Gentzsch - D-Grid Wolfgang Gerteis - SAP Belfast Doan Hoang - University of Technology, Sydney David Hutchison - Lancaster University Adriana Iamnitchi - University of South Florida Admela Jukan - Universite de Quebec Jussi Kangasharju - Technical University Darmstadt Gigi Karmous-Edwards - MCNC Dieter Kranzlmüller - University of Linz Joe Mambretti - Northwestern Univ Max Mühlhäuser - Technical University Darmstadt Nicholas Race - Lancaster University Matei Ripeanu - University of British Columbia Volker Sander - University of Applied Sciences Aachen Dimitra Simeonidou - University of Essex Oliver Yu - University of Illinois at Chicago

1 0