2019-08-05: Development meeting¶

Craig, Kacper, Elias, Mike H, Tommy

Agenda¶

Updates
Discussions
- v0.8 RC1
  - Before cutting RC, want to fix:
    - https://github.com/whole-tale/dashboard/issues/523
    - https://github.com/whole-tale/girderfs/pull/12
  - And review
    - https://github.com/whole-tale/wt-design-docs/pull/87
- Remote execution
  - Design document
  - Figure out tasking
  - Agave communication channels?
  - SAGA/Apache Airavata
    - https://saga-python.readthedocs.io/en/latest/index.html
    - https://cwiki.apache.org/confluence/display/AIRAVATA/Apache+Airavata+Overview
  - Also consider data locality? Ability to expose data known to be local without moving it
    - Globus: replica location service
    - DesignSafe and Cyverse data are on Agave-enable resources?
    - MH:
      - We seem to be branching out into grid computing space, which is a quagmire
      - We should provide services to allow users to submit their own jobs on batch resources, empowering them to figure out where to and how to run things.
    - Motivating use cases: need to be realistic and detailed

Updates¶

Craig
- v0.8 testing
  - Dashboard displays job as tale
  - Instance cap not handled during AiWT
- Remote execution – preplanning
- Kubernetes review – work in progress
- Paper revisions… (ParCo, RO)
Kacper:
- Notification modifications
- Meetings & PR review
Elias:
- Set up whole tale environment
- Submitted PR for Agave oauth file requests
  - https://github.com/whole-tale/girder_wt_data_manager/pull/43
  - https://github.com/whole-tale/girder_wholetale/pull/331
    - Remove WIP
    - Circle tests must pass
    - Update description to describe basic functional test
    - For example:
      - Step 1: Configure DesignSafe oauth and how
      - Step 2: Register using example DesignSafe DOI
      - Step 3: Browse files via UI and within contain
    - https://github.com/whole-tale/wt-design-docs/ -> https://wholetale.readthedocs.io/en/latest/
      - Design documentation (can be links a Google Doc or a readme in a repo)
      - https://wholetale.readthedocs.io/en/stable/development/design_notes/index.html
Mike H
- Looked into Agave stuff
- Proof-of-concept: Tale running on compute node via Singularity/Agave and routed back
  - Forward port from compute node to some other point (e.g., SSH tunnel) to Whole Tale
  - Automated tunneling in Agave?
Tommy
- 0.8 documentation updates
- Testing
  - https://github.com/whole-tale/girder_wholetale/issues/337
  - KK: Was looking for proper term for file size and MD5 in BagIt
- Feature request from Matt:
  - Reconstruct the filesystem paths when downloading an exported Tale from DataONE
  - Involves picking a vocab word from some vocabulary to describe the path and putting it in the resource map
  - Also involves modifying Metacat to parse the resource map when someone clicks Download All from MetacataUI.
  - I’m in the process of recommending schema:contentUrl (modifying their docs and then PR)
  - https://github.com/IQSS/dataverse/pull/3412/files#diff-4e738a204ca5caaa8bca95e54aee845e
- Q. CW
  - https://github.com/whole-tale/dashboard/issues/523?