Re: [SAGA-RG] SAGA Resource Package Comments
Andre, I like the generality that a resource provides services - however I take your point that it does make the users code more verbose. Perhaps getService(capability) would do the trick where capability is the set of capabilites that you require for the service. It provides at most one service. Steve On 16 January 2013 12:29, Shantenu Jha <shantenu.jha@rutgers.edu> wrote:
On 1/16/13 7:13 AM, Andre Merzky wrote:
On Wed, Jan 16, 2013 at 1:01 PM, Ole Weidner <ole.weidner@rutgers.edu> wrote:
Why not? I should be able to use a "network resource" to manage my bandwidth reservations / allocations, right?
A network resource is only useful if you can specify what resources it connects -- that is one of the uses we had for the resource pool, which I think won't exist anymore in the next version. So, how would you specify connectivity?
It is still early days in our understanding, but at some stage possibly, the ability to include networks as a resource will be important. We have not scoped this out yet, but my hope would be that the current design would be "extensible" to other/novel resources as eventually needed, whilst retaining simplicity to address the compute and storage resources that are well known at this stage.
Hi Steve, On Wed, Jan 16, 2013 at 2:33 PM, Steve Fisher <dr.s.m.fisher@gmail.com> wrote:
Andre,
I like the generality that a resource provides services - however I take your point that it does make the users code more verbose. Perhaps getService(capability) would do the trick where capability is the set of capabilites that you require for the service. It provides at most one service.
While this is indeed more compact and simpler, it is still redundant, isn't it? e.g., a compute resource will only ever offer a job service, etc. If we want to make that less redundant, and to make that generic getResource() call useful, we would need to reconsider the way we actually obtain resource handles in the first place -- and then possibly only have generic resource handles -- is that what you envision? Ole, any opinion on this one? Thanks, Andre.
Steve
On 16 January 2013 12:29, Shantenu Jha <shantenu.jha@rutgers.edu> wrote:
On 1/16/13 7:13 AM, Andre Merzky wrote:
On Wed, Jan 16, 2013 at 1:01 PM, Ole Weidner <ole.weidner@rutgers.edu> wrote:
Why not? I should be able to use a "network resource" to manage my bandwidth reservations / allocations, right?
A network resource is only useful if you can specify what resources it connects -- that is one of the uses we had for the resource pool, which I think won't exist anymore in the next version. So, how would you specify connectivity?
It is still early days in our understanding, but at some stage possibly, the ability to include networks as a resource will be important. We have not scoped this out yet, but my hope would be that the current design would be "extensible" to other/novel resources as eventually needed, whilst retaining simplicity to address the compute and storage resources that are well known at this stage.
-- There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton
If you create your resource by instantiating an image of a virtual machine then that resource may offer a variety of services. Even if you ask some kind of resource factory for a resource offering services with certain capabilities it may still include other services. Steve On 16 January 2013 13:47, Andre Merzky <andre@merzky.net> wrote:
Hi Steve,
On Wed, Jan 16, 2013 at 2:33 PM, Steve Fisher <dr.s.m.fisher@gmail.com> wrote:
Andre,
I like the generality that a resource provides services - however I take your point that it does make the users code more verbose. Perhaps getService(capability) would do the trick where capability is the set of capabilites that you require for the service. It provides at most one service.
While this is indeed more compact and simpler, it is still redundant, isn't it? e.g., a compute resource will only ever offer a job service, etc. If we want to make that less redundant, and to make that generic getResource() call useful, we would need to reconsider the way we actually obtain resource handles in the first place -- and then possibly only have generic resource handles -- is that what you envision? Ole, any opinion on this one?
Thanks, Andre.
Steve
On 16 January 2013 12:29, Shantenu Jha <shantenu.jha@rutgers.edu> wrote:
On 1/16/13 7:13 AM, Andre Merzky wrote:
On Wed, Jan 16, 2013 at 1:01 PM, Ole Weidner <ole.weidner@rutgers.edu> wrote:
Why not? I should be able to use a "network resource" to manage my bandwidth reservations / allocations, right?
A network resource is only useful if you can specify what resources it connects -- that is one of the uses we had for the resource pool, which I think won't exist anymore in the next version. So, how would you specify connectivity?
It is still early days in our understanding, but at some stage possibly, the ability to include networks as a resource will be important. We have not scoped this out
but my hope would be that the current design would be "extensible" to other/novel resources as eventually needed, whilst retaining simplicity to address
yet, the
compute and storage resources that are well known at this stage.
-- There are only two hard things in Computer Science: cache invalidation and naming things.
-- Phil Karlton
+1 That has been my argument for a looong time… separation of concerns. Resources don't necessarily expose a job submission or file transfer endpoint at all, hence job/file handling shouldn't be part of the resource API! - Ole On Jan 16, 2013, at 16:46 , Steve Fisher <dr.s.m.fisher@gmail.com> wrote:
If you create your resource by instantiating an image of a virtual machine then that resource may offer a variety of services. Even if you ask some kind of resource factory for a resource offering services with certain capabilities it may still include other services.
Steve
On 16 January 2013 13:47, Andre Merzky <andre@merzky.net> wrote: Hi Steve,
On Wed, Jan 16, 2013 at 2:33 PM, Steve Fisher <dr.s.m.fisher@gmail.com> wrote:
Andre,
I like the generality that a resource provides services - however I take your point that it does make the users code more verbose. Perhaps getService(capability) would do the trick where capability is the set of capabilites that you require for the service. It provides at most one service.
While this is indeed more compact and simpler, it is still redundant, isn't it? e.g., a compute resource will only ever offer a job service, etc. If we want to make that less redundant, and to make that generic getResource() call useful, we would need to reconsider the way we actually obtain resource handles in the first place -- and then possibly only have generic resource handles -- is that what you envision? Ole, any opinion on this one?
Thanks, Andre.
Steve
On 16 January 2013 12:29, Shantenu Jha <shantenu.jha@rutgers.edu> wrote:
On 1/16/13 7:13 AM, Andre Merzky wrote:
On Wed, Jan 16, 2013 at 1:01 PM, Ole Weidner <ole.weidner@rutgers.edu> wrote:
Why not? I should be able to use a "network resource" to manage my bandwidth reservations / allocations, right?
A network resource is only useful if you can specify what resources it connects -- that is one of the uses we had for the resource pool, which I think won't exist anymore in the next version. So, how would you specify connectivity?
It is still early days in our understanding, but at some stage possibly, the ability to include networks as a resource will be important. We have not scoped this out yet, but my hope would be that the current design would be "extensible" to other/novel resources as eventually needed, whilst retaining simplicity to address the compute and storage resources that are well known at this stage.
-- There are only two hard things in Computer Science: cache invalidation and naming things.
-- Phil Karlton
Sure, I can create a 'teleskope' resource which is neither a job submission endpoint nor a filesystem endpoint -- but that is not part of the use cases we considered for the resource API so far. The set of considered use cases is fairly limited actually, and that is, IMHO, a good thing (TM). And for those use cases, binding the resource type to a well defined SAGA service endpoint seems to work reasonably well. The intent is *not* to define a resource management interface for arbitrary resource types -- there are enough solutions for that. Could you come up with some (even a small number) of convincing and relevant use cases to motivate the resource API to be more generic and (slightly) less simple to use? Best, Andre. On Wed, Jan 16, 2013 at 6:06 PM, Ole Weidner <ole.weidner@rutgers.edu> wrote:
+1
That has been my argument for a looong time… separation of concerns.
Resources don't necessarily expose a job submission or file transfer endpoint at all, hence job/file handling shouldn't be part of the resource API!
- Ole
On Jan 16, 2013, at 16:46 , Steve Fisher <dr.s.m.fisher@gmail.com> wrote:
If you create your resource by instantiating an image of a virtual machine then that resource may offer a variety of services. Even if you ask some kind of resource factory for a resource offering services with certain capabilities it may still include other services.
Steve
On 16 January 2013 13:47, Andre Merzky <andre@merzky.net> wrote:
Hi Steve,
On Wed, Jan 16, 2013 at 2:33 PM, Steve Fisher <dr.s.m.fisher@gmail.com> wrote:
Andre,
I like the generality that a resource provides services - however I take your point that it does make the users code more verbose. Perhaps getService(capability) would do the trick where capability is the set of capabilites that you require for the service. It provides at most one service.
While this is indeed more compact and simpler, it is still redundant, isn't it? e.g., a compute resource will only ever offer a job service, etc. If we want to make that less redundant, and to make that generic getResource() call useful, we would need to reconsider the way we actually obtain resource handles in the first place -- and then possibly only have generic resource handles -- is that what you envision? Ole, any opinion on this one?
Thanks, Andre.
Steve
On 16 January 2013 12:29, Shantenu Jha <shantenu.jha@rutgers.edu> wrote:
On 1/16/13 7:13 AM, Andre Merzky wrote:
On Wed, Jan 16, 2013 at 1:01 PM, Ole Weidner <ole.weidner@rutgers.edu> wrote:
Why not? I should be able to use a "network resource" to manage my bandwidth reservations / allocations, right?
A network resource is only useful if you can specify what resources it connects -- that is one of the uses we had for the resource pool, which I think won't exist anymore in the next version. So, how would you specify connectivity?
It is still early days in our understanding, but at some stage possibly, the ability to include networks as a resource will be important. We have not scoped this out yet, but my hope would be that the current design would be "extensible" to other/novel resources as eventually needed, whilst retaining simplicity to address the compute and storage resources that are well known at this stage.
-- There are only two hard things in Computer Science: cache invalidation and naming things.
-- Phil Karlton
-- There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton
I am not even going to attempt it. I just find general solutions are easier to understand and work with than specific ones and are likely to have a longer life. Steve On 16 January 2013 19:00, Andre Merzky <andre@merzky.net> wrote:
Sure, I can create a 'teleskope' resource which is neither a job submission endpoint nor a filesystem endpoint -- but that is not part of the use cases we considered for the resource API so far. The set of considered use cases is fairly limited actually, and that is, IMHO, a good thing (TM). And for those use cases, binding the resource type to a well defined SAGA service endpoint seems to work reasonably well. The intent is *not* to define a resource management interface for arbitrary resource types -- there are enough solutions for that.
Could you come up with some (even a small number) of convincing and relevant use cases to motivate the resource API to be more generic and (slightly) less simple to use?
Best, Andre.
+1
That has been my argument for a looong time… separation of concerns.
Resources don't necessarily expose a job submission or file transfer endpoint at all, hence job/file handling shouldn't be part of the resource API!
- Ole
On Jan 16, 2013, at 16:46 , Steve Fisher <dr.s.m.fisher@gmail.com> wrote:
If you create your resource by instantiating an image of a virtual machine then that resource may offer a variety of services. Even if you ask some kind of resource factory for a resource offering services with certain capabilities it may still include other services.
Steve
On 16 January 2013 13:47, Andre Merzky <andre@merzky.net> wrote:
Hi Steve,
On Wed, Jan 16, 2013 at 2:33 PM, Steve Fisher <dr.s.m.fisher@gmail.com> wrote:
Andre,
I like the generality that a resource provides services - however I
take
your point that it does make the users code more verbose. Perhaps getService(capability) would do the trick where capability is the set of capabilites that you require for the service. It provides at most one service.
While this is indeed more compact and simpler, it is still redundant, isn't it? e.g., a compute resource will only ever offer a job service, etc. If we want to make that less redundant, and to make that generic getResource() call useful, we would need to reconsider the way we actually obtain resource handles in the first place -- and then possibly only have generic resource handles -- is that what you envision? Ole, any opinion on this one?
Thanks, Andre.
Steve
On 16 January 2013 12:29, Shantenu Jha <shantenu.jha@rutgers.edu> wrote:
On 1/16/13 7:13 AM, Andre Merzky wrote:
On Wed, Jan 16, 2013 at 1:01 PM, Ole Weidner <
On Wed, Jan 16, 2013 at 6:06 PM, Ole Weidner <ole.weidner@rutgers.edu> wrote: ole.weidner@rutgers.edu>
wrote: > > Why not? I should be able to use a "network resource" to manage my > bandwidth > reservations / allocations, right?
A network resource is only useful if you can specify what resources it connects -- that is one of the uses we had for the resource pool, which I think won't exist anymore in the next version. So, how would you specify connectivity?
It is still early days in our understanding, but at some stage possibly, the ability to include networks as a resource will be important. We have not scoped this out yet, but my hope would be that the current design would be "extensible" to other/novel resources as eventually needed, whilst retaining simplicity to address the compute and storage resources that are well known at this stage.
-- There are only two hard things in Computer Science: cache invalidation and naming things.
-- Phil Karlton
-- There are only two hard things in Computer Science: cache invalidation and naming things.
-- Phil Karlton
participants (3)
-
Andre Merzky -
Ole Weidner -
Steve Fisher