In the course of implementing some additional JSDL 1.0 features (or
extensions), a couple of issues have reared their ugly heads that I
wanted to bring to the group's attention.
The first has to do with data staging, working directory, file systems,
and everything else having to do with paths. It turns out that there
are some exceptionally popular applications out there that are very
poorly written. This is probably a surprise to no one, but
unfortunately, its a fact of life that in the grid computing business,
we have to support many of these applications despite their
egregiousness. In particular, I refer to applications which cannot
handle input files or paths which contain spaces. Recently, while
trying to run a large batch of BLAST jobs using our new implementation
of the SCRATCH file system, we discovered that BLAST is unable to deal
with paths to the search database which contain spaces. As we thought
about what to do about this, we realized that, sad as it may seem, it
may be necessary to add to JSDL the ability to say that an Application
(or certain inputs) must have paths without any spaces included. Would
this be a Resource requirement? I'm not sure exactly how to classify
it, but it seems like some JSDL annotation is necessary.
The second problem has to do with the SPMD extensions. I feel very
strongly that the enumerated type for which kind of SPMD application
you are is insufficient. If you look at UVa as an example compute
environment, we have at least 3 versions of MPICH running on our cluster
at any given time (one each for the gcc, pgi, and intel compilers).
Each one requires different libraries to be located in the search path
and each one is started (sadly) with a different mpiexec. I think that
a rework of the SPMD extensions to allow for some kind of greater
specificity in the SPMD type is called for.
Sincerely,
Mark Morgan
