2018-11-05: Development meeting

Kacper, Craig, Damian, Mike L, Bryce, Adam, Kyle, Mike H, Ian

Agenda

  • Updates
  • Priorities:
    • “Analyze in WT”
      • Backend Import / “Ad hoc” tale creation (KK)
      • UI (TT)
      • DataONE dev instance + button (TT)
        • Consider deploying DataONE dev instance (CW/BM)
    • Dataverse integration
      • Implement Dataverse register (KK – 1 wk)
      • External tools JSON
      • Dataverse dev instance + button (CW)
    • Release (v0.5?)
      • BM: Would need canned data for button to work with DataONE
    • Tale serialization format refactor
    • Workspace support
      • Finalize mockups/define tasks
      • Backend (done - KK)
      • UI (ND – starting in December)
      • Publish integration (TT)
    • Publish
      • To DataONE (dev instance only for now)
      • To Zenodo (no barrier – 2 weeks)
    • Release (v0.6)
    • Add/remove data
      • Finalize mockups/define tasks
      • Backend (done – MH)
      • UI (ND – starting in December)
    • Custom environment
      • Design
      • repo2docker integration
      • UI TBD (when to rebuild images, notifications, switching environments)
    • Copy on Launch
      • Finalize mockups
      • Backend
      • UI refactor
    • Other tasks
      • Refactor view/edit Tale metadata
      • Compose modal
      • Move “Home” capability to new “Run” page
      • Move environments to browse/environments page
      • Create browse/catalog page
      • Tale validation
      • Saving tales
      • Real tales (Seni, Kyle B)
      • Automated testing
      • Migrate to Kubernetes
  • Releases
    • v0.5 tentatively end of November
    • v0.6 Januaryish release

Updates

  • Kacper:
    • Introducting Mike Lambert
    • Worked on import tale/create ad-hoc tale
    • Tested Mike H.’s PR, but waiting on additional privileges (Globus+). Would like to merge this ASAP
      • KC: Checkin in
    • Will use the refactored provider model to implement Dataverse register
    • Spent time on UI prioritization
  • Mike L.
    • Got dev environment running
    • Ramping up on Ember JS
    • Q. Discrepencies in Ember styling in development and production environments, maybe caused by legacy class that is only in production
      • Was able to replicate by passing production to Ember serve
      • KK: dev and production WT run Ember production environment
  • Ian/Adam/Damian
    • Reviewed UI tasks.
    • Damian/Adam provided high-level estimateion for ~5 weeks on workspace and add/remove data support
    • Will likely start work in December with 2 people working over 3 weeks
    • Sprint reviews Monday mornings
  • Bryce:
    • Last week at IEEE eScience, talk on DataONE packaging format
    • Everyone is doing WT/Binder like stuff
      • Swan - CERN; Stencila, etc
    • NextJournal – custom notebook environment (not Jupyter)
    • Going to AGU (12/10-14, Washington , DC)
    • Tale serialization format:
      • Matt J wants a single-file BagIt/Zip serialization
      • Had a dream if the tale.yaml could be used to materialize things locally, but MJ wanted a relatively-fully-materialized. BagIt is one way to do this.
      • BagIt allows you to not materialize all of the data (e.g., what if there’s a 10TB file)
      • KC: This is a good idea. BigDataBag has a profile over the BagIt format we should consider.
        • RO metadata model may be too heavyweight
      • RDA has a profile repository
      • KC: How we’re linking in data here could be used to augment BagIt fetch model. Whole Tale could do this, BagIt doesn’t currently do it.
      • KK: DataLad does something similar. There are people interested in tapping Git large filesystem in this space (LSST).
    • Will arrange/participate on call to discuss.
  • Mike H.
    • Working on design document about Tales and copy and write.
    • KK: Is next step add/remove files from running sessions?
      • MH: This is done. Needs to be tested, but otherwise available for UI work.
  • Tommy:
    • Created a branch and committed code for handling someone coming from a third party with the intent of registering&launching a tale w/ data.
    • Discussed using RDF to represent the information in the contents of tale.yaml
      • This week I’ll be putting a document together of how this would look
    • Along the same lines, we discussed prov (how to store it, how to collect it)
      • We may be able to use the prov model for validating runs-each run creates a
    • Migrated publishing code to the publishing branches
      • Today I’ll be doing more testing and putting some ideas about what to do with the file picker (do we only show files)
    • KK: Q. Why to we need the base_url?
      • The member node doesn’t know which coordinating node to talk to.
    • Check out the PR where data registration was re-factored
  • Craig:
    • Development planning
    • RDA fellows
    • Carpentry workshop