Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10812

Enumerate DRP data products and estimate storage and possibly recompute costs

    XMLWordPrintable

Details

    • Story
    • Status: To Do
    • Resolution: Unresolved
    • None
    • Design Documents
    • None
    • 6
    • Data Release Production

    Description

      These estimates are needed as a near-term refresh of the sizing model.

      I'm planning to do this initially as a spreadsheet of all planned DRP data products (including temporaries), with annotations for how they relate to user-accessible data products in the DPDD.

      I'm sure I'll come up with lots of questions about how to make this information most useful to its consumers (primarily NCSA, I think), and I'll post those in the comments here.

      Attachments

        Issue Links

          Activity

            tjenness Tim Jenness added a comment -

            Can I assume that this size refresh for DRP has long been done else USDF proposals would not be possible?

            tjenness Tim Jenness added a comment - Can I assume that this size refresh for DRP has long been done else USDF proposals would not be possible?
            ktl Kian-Tat Lim added a comment -

            A version of this was done by determining the actual sizes of broad categories of DRP data products based on the RC2 processing after elimination of obvious temporaries, without further enumeration or prediction of future products.  An allocation was made for future growth.

            But a detailed accounting could still be useful.

            ktl Kian-Tat Lim added a comment - A version of this was done by determining the actual sizes of broad categories of DRP data products based on the RC2 processing after elimination of obvious temporaries, without further enumeration or prediction of future products.  An allocation was made for future growth. But a detailed accounting could still be useful.
            jbosch Jim Bosch added a comment -

            Without a clear deadline, this is always going to be a victim of both the fact that it's a lot of work and "if we wait the result will be more future-proof, because we know X, Y, and Z are due to change anyway". If we want to set one soon-ish because we think it's a useful thing to force us to do, I think a good time would be during the execution of DP0.2. But if no one will really pay too much attention to the result, I'd be more inclined to just close this ticket and open a new one if/when a clear need aries.

            jbosch Jim Bosch added a comment - Without a clear deadline, this is always going to be a victim of both the fact that it's a lot of work and "if we wait the result will be more future-proof, because we know X, Y, and Z are due to change anyway". If we want to set one soon-ish because we think it's a useful thing to force us to do, I think a good time would be during the execution of DP0.2. But if no one will really pay too much attention to the result, I'd be more inclined to just close this ticket and open a new one if/when a clear need aries.

            People

              jbosch Jim Bosch
              jbosch Jim Bosch
              Jim Bosch, Kian-Tat Lim, Margaret Gelman, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:

                Jenkins

                  No builds found.