2020-10-05: Development Meeting

Craig, Tommy, Kacper, Tim, Mike L., Mike H

Agenda

  • Updates
  • C2Metadata integration Demo/workflow discussion – [name=Tommy]
    • Now that SDTL isn’t in the recorded run, barriers for a demo are much lower, we should chat about it
    • What would a demo look like?
      • Input dataset with DDI or assume “generic” DDI generation for arbitrary
      • Run SDTL tools via BrownDog (ostensibly via recorded run)
        • Tommy: Could do this during publish?
      • Q. Do they have an EML parser?
        • CW: I don’t think so
      • Publish into DataONE
        • Tommy: prospective provenance is almost done
        • Process works with SDTL json file > RDF for resource map
      • Do we show visualization?
        • Tommy: Not sure DataONE will process this Prov
          • May assume prov is on the file level
    • Tim:
      • What about a plugin model?
      • Or a way to call things like SDTL or YW before/after run.sh
      • Don’t want YW or SDTL to be traced during recorded run.

Updates

  • Craig
    • PR Review
    • Proof-of-concept repo2docker with MATLAB support
      • repo2docker modifications to support buildkit, including using subprocess instead of docker-py
      • repo2docker modifications to support passing build-args
      • Preliminary matlab buildkit using RUN --mount...
    • Really need to get back into C2Metadata/BD integration…
    • Also XSEDE XRAC
  • Kacper
    • Git integration
      • missing tests, should be ready for reviewes this week
    • Planning on merging virt_resources changes and pushing versioning next.
    • Also XSEDE XRAC
  • Tommy
    • Not too much WT work last week
    • Reviewing PRs right now
    • Next up:
      • Prepping for next C2Metadata meeting this Thursday
      • Finish/reorganize up that stale publishing PR
      • Spec out C2Metadata + Publishing behavior
      • Spec out the identifier scheme for feedback(partially complete)
  • Tim
    • Adapted Comprehensive Provenance Record (CPR) to output RDF triples in addition to facts.
    • Still working on vocabulary and IDs. This will continue to evolve as work on queries and visualizations proceeds.
    • The triples can be loaded into Blazegraph. Will add option to CPR to load trace directly.
  • Mike H. *
  • Mike L.
    • PR for improvements to error-handling in publishing
    • Fixed some merge conflicts in older PRs
      • Apparently WIP PRs won’t update display when conflicts are fixed - needed to mark “Ready for Review”