2019-08-05: Development meeting¶
Craig, Kacper, Elias, Mike H, Tommy
Agenda¶
- Updates
- Discussions
- v0.8 RC1
- Before cutting RC, want to fix:
- https://github.com/whole-tale/dashboard/issues/523
- https://github.com/whole-tale/girderfs/pull/12
- And review
- https://github.com/whole-tale/wt-design-docs/pull/87
- Before cutting RC, want to fix:
- Remote execution
- Design document
- Figure out tasking
- Agave communication channels?
- SAGA/Apache Airavata
- https://saga-python.readthedocs.io/en/latest/index.html
- https://cwiki.apache.org/confluence/display/AIRAVATA/Apache+Airavata+Overview
- Also consider data locality? Ability to expose data known to be local without moving it
- Globus: replica location service
- DesignSafe and Cyverse data are on Agave-enable resources?
- MH:
- We seem to be branching out into grid computing space, which is a quagmire
- We should provide services to allow users to submit their own jobs on batch resources, empowering them to figure out where to and how to run things.
- Motivating use cases: need to be realistic and detailed
- v0.8 RC1
Updates¶
- Craig
- v0.8 testing
- Remote execution – preplanning
- Kubernetes review – work in progress
- Paper revisions… (ParCo, RO)
- Kacper:
- Notification modifications
- Meetings & PR review
- Elias:
- Set up whole tale environment
- Submitted PR for Agave oauth file requests
- https://github.com/whole-tale/girder_wt_data_manager/pull/43
- https://github.com/whole-tale/girder_wholetale/pull/331
- Remove WIP
- Circle tests must pass
- Update description to describe basic functional test
- For example:
- Step 1: Configure DesignSafe oauth and how
- Step 2: Register using example DesignSafe DOI
- Step 3: Browse files via UI and within contain
- https://github.com/whole-tale/wt-design-docs/ -> https://wholetale.readthedocs.io/en/latest/
- Design documentation (can be links a Google Doc or a readme in a repo)
- https://wholetale.readthedocs.io/en/stable/development/design_notes/index.html
- Mike H
- Looked into Agave stuff
- Proof-of-concept: Tale running on compute node via Singularity/Agave and routed back
- Forward port from compute node to some other point (e.g., SSH tunnel) to Whole Tale
- Automated tunneling in Agave?
- Tommy
- 0.8 documentation updates
- Testing
- https://github.com/whole-tale/girder_wholetale/issues/337
- KK: Was looking for proper term for file size and MD5 in BagIt
- Feature request from Matt:
- Reconstruct the filesystem paths when downloading an exported Tale from DataONE
- Involves picking a vocab word from some vocabulary to describe the path and putting it in the resource map
- Also involves modifying Metacat to parse the resource map when someone clicks
Download All
from MetacataUI. - I’m in the process of recommending
schema:contentUrl
(modifying their docs and then PR) - https://github.com/IQSS/dataverse/pull/3412/files#diff-4e738a204ca5caaa8bca95e54aee845e
- Q. CW
- https://github.com/whole-tale/dashboard/issues/523?