Hiro Kishimoto wrote:
Hi all,
Jun Tatemura and I took a home work assignment about "Basic POSIX Component" for BLAST application deployment.
Attached is my first-cut draft and hope to discuss at GGF17 OGSA EMS session. Please have a look and give your feedbacks before or at the GGF17. Your feedbacks are very important.
Thanks and see you in Tokyo!
This is really good, and I do agree, we do need a set of foundational things to deploy. One thing I would argue, having learned the lessons of both Ant and SmartFrog, is that a consistent model of file system components comes first. That is, we need components to model -files (with a liveness test that verifies the file is present) -paths (an ordered list of files/directories) -directories (with components that can create a dir on deployment, optionally to delete it and its contents on termination) -wildcarded sets of files (**/*.csv) -temporary files and directories -cached downloads -text files I'm attaching the PDF file describing the current smartfrog set of components that do this. The underlying way that this is done is that all components that consider themselves to be sources of filenames/paths set the attribute absolutePath at runtime. They usually also export an RMI interface for RPC-style operations, along side their RESTy state. A standard set of filesystem components lets you include the operations to create and clean up directories, temporary files and other housekeeping operations into the workflows of a deployment. Looking at the Posix proposal, I'd suggest starting with the reusable set of filesystem types, including a <fs:mount> component that could mount a local or remote filesystem, and which would then pass the local location to interested parties as the filesystem:absolutePath attribute. You absolutely don't need to introduce a new shared datatype FileSystemName, because that implies some kind of alternate naming/locating scheme outside of what CDL and the runtime has built in. Dynamic CDL references are sufficient to crosslink information inside the graph of deployed things, and offer flexibility as well as ease of management (you see them when you walk the graph). First, we'd need filesystem types: <tmp cdl:extends="fs:TempFileSystem"> <fs:Description> … </fs:Description> <fs:DiskSpace> <fs:LowerBoundedRange>10737418240.0</fs:LowerBoundedRange> </fs:DiskSpace> </tmp> <home cdl:extends="fs:Directory"> <fs:Description>Chris's home directory</fs:Description> <fs:dir>/home/csmith</fs:dir> </home> When deployed, both of these would add absolutePath as an attribute, so that tmp/absolutePath woudl resolve to, say, tmp/work01234/ and home:absolutePath to /home/csmith. You'd then mount a blast dir, say on an nfs filesys whose URL you provide <blastfs cdl:extends="fs:RemoteFileSystem"> <fs:url>nfs://filestore/csmith/blast</fs:url> <fs:mountUser>csmith</fs:mountUser> </tmp> The database file is under here, and you declare that it must exist. Deployment will fail if it is absent: <db cdl:extends="fs:File"> <fs:dir cdl:ref="/tmp/absolutePath" cdl:lazy="true" /> <fs:filename>db/ncbiblast/est</fs:filename> <fs:filemustexist>true</fs:filemustexist> </db> To use it, just refer to its path <blast:database cdl:ref="/db/absolutePath" cdl:lazy="true"/> Paths could be represented by some list-like element, whose entries are evaluated at deploy time and then a platform-specific path created as a result <path cdl:extends="fs:Path"> <pathentry cdl:ref="/blastfs/absolutePath" cdl:lazy="true" /> <pathentry cdl:ref="/home/absolutePath" cdl:lazy="true"/> </path> On Windows /path/absolutePath would become something like "\\filestore\csmith\blast;c:\documents and settings\csmith" while on unix it could be "/nfs/filestore/csmith/blast:/home/csmith" This information now goes down to the jsdl component <jsdl-posix:Environment name="PATH" cdl:ref="path:absolutePath" cdl:lazy="true"/> <jsdl-posix:Environment name="TMPDIR" cdl:ref="tmp/absolutePath" cdl:lazy="true"/> -Steve