Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15059

Investigate and report on the impact to the DM design to keep lossy-compressed PVIs on disk

    Details

    • Type: Improvement
    • Status: In Review
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: Design Documents
    • Labels:
      None
    • Team:
      Data Facility

      Description

      The Lossy Compression Working Group's work (DM-11819, DMTN-068) suggests that it may be realistic and useful to maintain an on-disk archive of at least one data release's PVIs for all visits, together with the AP-derived PVIs for more recent data that's not been through a data release yet.

      We now need a cost and design impact study for doing this, in order to be ready to process DM- and Project-level change requests to add this to the baseline.

      This should be considered together with DM-11880, the request to clarify whether we have already added resources for the storage of all raw image data on disk to the project baseline.

        Attachments

          Issue Links

            Activity

            Hide
            gpdf Gregory Dubois-Felsmann added a comment -

            Marked as "critical" because deciding on this is a blocker for adding image access performance requirements to LSE-61 (the DM System Requirements document).

            Show
            gpdf Gregory Dubois-Felsmann added a comment - Marked as "critical" because deciding on this is a blocker for adding image access performance requirements to LSE-61 (the DM System Requirements document).
            Hide
            mgelman2 Margaret Gelman added a comment -

            I plan to fold this data retention policy change into the impending refactored sizing model.

            Show
            mgelman2 Margaret Gelman added a comment - I plan to fold this data retention policy change into the impending refactored sizing model.
            Hide
            mbutler Michelle Butler added a comment -

            Here is a spread sheet for what keeping the PVI's for the current release on disk, and then also keep the PVI's of the current "year" as it's not in a release yet, but will be.   

            https://docs.google.com/spreadsheets/d/1M-5xwaZNeIs21Z5UN3T9w2S-EvLvdCbqDxe8XLMtkqQ/edit?usp=sharing

            Show
            mbutler Michelle Butler added a comment - Here is a spread sheet for what keeping the PVI's for the current release on disk, and then also keep the PVI's of the current "year" as it's not in a release yet, but will be.    https://docs.google.com/spreadsheets/d/1M-5xwaZNeIs21Z5UN3T9w2S-EvLvdCbqDxe8XLMtkqQ/edit?usp=sharing
            Hide
            mbutler Michelle Butler added a comment -

            added a spread sheet for how much keeping 1 years of PVI's for a release on disk, and keeping the current "being built" release also on disk.    

            FYI 

            M. 

             

            Show
            mbutler Michelle Butler added a comment - added a spread sheet for how much keeping 1 years of PVI's for a release on disk, and keeping the current "being built" release also on disk.     FYI  M.   

              People

              • Assignee:
                gpdf Gregory Dubois-Felsmann
                Reporter:
                gpdf Gregory Dubois-Felsmann
                Reviewers:
                Gregory Dubois-Felsmann
                Watchers:
                Fritz Mueller, Gregory Dubois-Felsmann, Kian-Tat Lim, Margaret Gelman, Michelle Butler, Tim Jenness, Wil O'Mullane
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Summary Panel