2019-01-02: Development Planning

Kacper, Craig

Development planning

  • v0.5 released to production, but announcement delayed pending Jetstream/IU deployment problems
    • PR issued to Dataverse but missed 4.10 release (https://github.com/IQSS/dataverse/pull/5417)
  • v0.6 (tentative)
    • Use case: create a blank tale, add data, upload/create notebook, make public. Another user can run but not edit.
    • Workspace support
    • Ability to select data and modify session (2w)
      • Requires switch from using copy folders and embrace dm
      • Today sessions are created for instances, but should be created for Tales
    • Fix directory structure
      • Requires backend changes and testing
    • Export?
  • v0.7 (tentative)
    • Publish/release/immutability
    • Copy on Launch
    • Files and metadata should work regardless of container state
    • Custom environment

Discussion of sessions

  • For v0.6, we need to embrace session API
  • This raises the question of which copy of data we are using to populate the session (i.e. original catalog, copy user has in data directory)
  • Recapping what we do right now:
    • When you register data, we have a flat namespace, we create an entry
    • If it’s already registered, we copy it to your data directory
    • Registration creates two entries – one in the Global catalog, one in your personal space
    • In current system, the user can reorganize and rename data in their personal space, but not global catalog
  • When you create the tale, which should we reference – global or your copy?
    • The session should operate on the catalog directly – there is no copying data
    • We would still need some way to differentiate between two users that regisitered the same dataset
    • Instead of having a folder structure per user, we could have the user model have a list of registered datasets
  • There will be a migration problem, we need to analyze
  • Discussed deletion
    • We need the ability to delete datasets directly (DELETE /dataset/{id})
    • We need the ability to implicitly delete/prune datasets during Tale deletion (DELETE /tale/{id}?prune_ds=true).
      • This should be non-fatal.
    • Deletion cases:
      • User registers dataset then deletes reference to dataset (likely via future “My Catalog” page)
      • User registers dataset, creates Tale
        • Delete dataset fails – tale is referencing it
        • Delete Tale prompts to ask if the data should also be deleted (Default yes)
      • User registers dataset, creates Tales 1 and 2
        • Delete dataset fails – tale is referencing it
        • Delete Tale 2 there is no prompt to delete data since it is referenced.
        • Delete Tale 2 then Tale 1, user is prompted