Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-24254

pipe_analysis reads calexps unnecessarily

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_analysis
    • Labels:
      None
    • Team:
      Architecture

      Description

      It appears that many (if not all) scripts in pipe_analysis don't actually use calexp or coadd pixels even though they read those datasets, sometimes several times. Replacing these by calexp_md, camera, calexp_wcs, or calexp_photoCalib wherever possible, or reading single pixel subimages if necessary, should reduce the disk read load considerably.

        Attachments

          Issue Links

            Activity

            Hide
            ktl Kian-Tat Lim added a comment -

            https://github.com/lsst-dm/pipe_analysis/pull/50

            There are no tests, and this isn't run by any CI job, so it's hard for me to check that I haven't broken anything.

            Show
            ktl Kian-Tat Lim added a comment - https://github.com/lsst-dm/pipe_analysis/pull/50 There are no tests, and this isn't run by any CI job, so it's hard for me to check that I haven't broken anything.
            Hide
            sophiereed Sophie Reed added a comment -

            It looks good to me and anything that speeds it up will be welcomed.

            Show
            sophiereed Sophie Reed added a comment - It looks good to me and anything that speeds it up will be welcomed.

              People

              Assignee:
              ktl Kian-Tat Lim
              Reporter:
              ktl Kian-Tat Lim
              Reviewers:
              Sophie Reed
              Watchers:
              Kian-Tat Lim, Lauren MacArthur, Sophie Reed
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.