2018-08-13: Development meeting

Kacper, Tommy, Damian, Sebastian, Mike, Craig, Ian

Agenda

  • Updates
  • Tasks that need to be done before 36mo (Craig). Based on PEP.
    • Search, Storage APIs within Jupyter, RStudio, etc
      • Will be obsoleted by adding launch functionality from Globus Publish and DataONE?
        • “Play in WT” button prototyped (register data and bring to compose page). TT: Branch with route in newui. Need to revisit – prior work is stale. (Medium risk: no backend support)
    • OAI-ORE filesystem
      • https://bitbucket.org/data-exp-lab/ideals-browser/src/default/
      • Nice to have.
    • Identifier issuing and resolution, including Globus Publication
      • Done via DataONE?
      • Globus has a service to mint identifiers (minid) as an alternative.
    • Document, store, publish in DataONE (Tommy) (v0.5)
      • Help needed with:
        • File Browser (tentatively: Adam, will meet Friday)
        • Getting D1 JWT more smoothly (tentatively: Kacper)
        • Run page top widget (Bryce)
        • Kacper to help with getting local environment running and gwvolman components.
      • Testing: UI is incomplete and notifications. Feedback on package itself welcome.
    • Globus ingestion wrappers (Mike)
      • 36 mo deliverable. MDF only as proof-of-concept that WT can support other indices.
    • Dashboard blockers:
  • Tasks that we would like to have asap but should be delivered in PY4-5
    • Indexing, remixing of frontends (Needs UI (Proposal D5.1.1 indicates Y4 deliverable)
      • Need UI team to provide estimates
  • Automated testing
    • Postman + Selenium
  • Who’s going where?

Updates

  • Kacper:

    • Roadmap
    • OpenRefine

  • Bryce:

    • Looked into why data registration is failing

  • Craig:

    • Reviewed the project requirements

  • Mike:

    • Registering data from Globus, refactoring registration framework

  • Tommy:

    • Investigated 1/13th file dying during dataone upload. (Fixed this morning)
    • Fixed the issue where files above 1mb cannot be uploaded to DataONE
    • Pushed code that supports publishing tales with data from DataONE
    • Fixed a bug where prod was using cn.dataone for user’s jwt
    • Updated ui so that it’s 1-1 with the backend
    • Pulled publishing stuff out of the repository and created a separate endpoint, /publish
    • Checked the status of globus-dataone integration. Still deadlocked

    Backend Publishing Status: Publishing works with packages that have DataONE data (Try the Jupyter Tale Demo Tale) Currently investigating a bug with tales with local files Porting to gwvolman so we can use the job watcher Frontend Publishing Status: Asked Bryce to help with the compenent on the Run page Need to fix OAuth popup (letting the modal know the user signed in-workaround is to refresh the page after the login) Make an issue for warning the user that, if they have files above 1GB, they won’t be accepted

  • Sebastian & Damian: