Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29407

visitSummary files should not persist the detector

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_tasks
    • Labels:
    • Story Points:
      1
    • Epic Link:
    • Sprint:
      DRP S21a (Dec Jan)
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      The visitSummary files currently persist the detector for each row, but it turns out that this makes the files 140x larger than otherwise, and take 50x longer to read than otherwise. Persisting the detector in here is definitely not necessary; all we need is the id and that is persisted as the row index.

        Attachments

          Activity

          Ascending order - Click to sort in descending order
          Hide
          Permalink
          erykoff Eli Rykoff added a comment -

          PRs are:
          https://github.com/lsst/jointcal/pull/175
          https://github.com/lsst/pipe_tasks/pull/488
          https://github.com/lsst/testdata_jointcal/pull/34
          Jenkins (including ci_hsc):
          https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/33886/pipeline

          Show
          erykoff Eli Rykoff added a comment - PRs are: https://github.com/lsst/jointcal/pull/175 https://github.com/lsst/pipe_tasks/pull/488 https://github.com/lsst/testdata_jointcal/pull/34 Jenkins (including ci_hsc): https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/33886/pipeline
          Hide
          Permalink
          Parejkoj John Parejko added a comment -

          Thanks for digging into this. You jointcal change looks good, baring one minor doc typo.

          Show
          Parejkoj John Parejko added a comment - Thanks for digging into this. You jointcal change looks good, baring one minor doc typo.

            People

            Assignee:
            erykoff Eli Rykoff
            Reporter:
            erykoff Eli Rykoff
            Reviewers:
            John Parejko
            Watchers:
            Eli Rykoff, John Parejko
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.