2019-08-05: Development meeting

Craig, Kacper, Elias, Mike H, Tommy

Agenda

  • Updates
  • Discussions
    • v0.8 RC1
      • Before cutting RC, want to fix:
        • https://github.com/whole-tale/dashboard/issues/523
        • https://github.com/whole-tale/girderfs/pull/12
      • And review
        • https://github.com/whole-tale/wt-design-docs/pull/87
    • Remote execution
      • Design document
      • Figure out tasking
      • Agave communication channels?
      • SAGA/Apache Airavata
        • https://saga-python.readthedocs.io/en/latest/index.html
        • https://cwiki.apache.org/confluence/display/AIRAVATA/Apache+Airavata+Overview
      • Also consider data locality? Ability to expose data known to be local without moving it
        • Globus: replica location service
        • DesignSafe and Cyverse data are on Agave-enable resources?
        • MH:
          • We seem to be branching out into grid computing space, which is a quagmire
          • We should provide services to allow users to submit their own jobs on batch resources, empowering them to figure out where to and how to run things.
        • Motivating use cases: need to be realistic and detailed

Updates

  • Craig
  • Kacper:
  • Elias:
    • Set up whole tale environment
    • Submitted PR for Agave oauth file requests
      • https://github.com/whole-tale/girder_wt_data_manager/pull/43
      • https://github.com/whole-tale/girder_wholetale/pull/331
        • Remove WIP
        • Circle tests must pass
        • Update description to describe basic functional test
        • For example:
          • Step 1: Configure DesignSafe oauth and how
          • Step 2: Register using example DesignSafe DOI
          • Step 3: Browse files via UI and within contain
        • https://github.com/whole-tale/wt-design-docs/ -> https://wholetale.readthedocs.io/en/latest/
          • Design documentation (can be links a Google Doc or a readme in a repo)
          • https://wholetale.readthedocs.io/en/stable/development/design_notes/index.html
  • Mike H
    • Looked into Agave stuff
    • Proof-of-concept: Tale running on compute node via Singularity/Agave and routed back
      • Forward port from compute node to some other point (e.g., SSH tunnel) to Whole Tale
      • Automated tunneling in Agave?
  • Tommy
    • 0.8 documentation updates
    • Testing
      • https://github.com/whole-tale/girder_wholetale/issues/337
      • KK: Was looking for proper term for file size and MD5 in BagIt
    • Feature request from Matt:
      • Reconstruct the filesystem paths when downloading an exported Tale from DataONE
      • Involves picking a vocab word from some vocabulary to describe the path and putting it in the resource map
      • Also involves modifying Metacat to parse the resource map when someone clicks Download All from MetacataUI.
      • I’m in the process of recommending schema:contentUrl (modifying their docs and then PR)
      • https://github.com/IQSS/dataverse/pull/3412/files#diff-4e738a204ca5caaa8bca95e54aee845e
    • Q. CW
      • https://github.com/whole-tale/dashboard/issues/523?