Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33875

Create LATISS SDSSr and SDSSi flats

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Team:
      DM Science

      Description

      Create LATISS SDSSr and SDSSi flats using the data from the Feb 2022 run and add them to the LATISS/calib chained collection

        Attachments

          Issue Links

            Activity

            Hide
            edennihy Erik Dennihy added a comment -

            Generated the collections LATISS/tickets/DM-33875 at NCSA, it includes SDSSr and SDSSi flats. Flats were verified through notebook visualizations. Processing from raw should only require adding LATISS/calib/DM-33875 to the list of standard input collections for AuxTel data processing.

            NOTE: The SDSSi flats are a composite of only 10 images taken on 2022-02-08. This should be replaced by a more extensive set in the future when available for higher quality flats.

            Show
            edennihy Erik Dennihy added a comment - Generated the collections LATISS/tickets/ DM-33875 at NCSA, it includes SDSSr and SDSSi flats. Flats were verified through notebook visualizations. Processing from raw should only require adding LATISS/calib/ DM-33875 to the list of standard input collections for AuxTel data processing. NOTE: The SDSSi flats are a composite of only 10 images taken on 2022-02-08. This should be replaced by a more extensive set in the future when available for higher quality flats.
            Hide
            czw Christopher Waters added a comment -

            These flats look fine to me as well.  To make them available for use, you should add them to the main LATISS calib collection:

            butler collection-chain /repo/main --mode=prepend LATISS/calib LATISS/calib/DM-33875 

            Next, to synchronize the summit, you'll need to export them:

            butler export-calibs /repo/main ./calibs.dm33875 LATISS/calib/DM-33875 

            Copy that directory to the summit, and then ingest and update the summit collection as well:

            butler import --export-file ./calibs.dm33875/export.yaml --transfer copy $REPO_SUMMIT ./calibs.dm33875/ -s instrument -s detector -s physical_filter
             
            butler collection-chain $REPO_SUMMIT --mode=prepend LATISS/calib LATISS/calib/DM-33875 
            

            When that's done, I'm happy to do a quick check and review.

             

            Show
            czw Christopher Waters added a comment - These flats look fine to me as well.  To make them available for use, you should add them to the main LATISS calib collection: butler collection-chain /repo/main --mode=prepend LATISS/calib LATISS/calib/DM- 33875 Next, to synchronize the summit, you'll need to export them: butler export-calibs /repo/main ./calibs.dm33875 LATISS/calib/DM- 33875 Copy that directory to the summit, and then ingest and update the summit collection as well: butler import --export-file ./calibs.dm33875/export.yaml --transfer copy $REPO_SUMMIT ./calibs.dm33875/ -s instrument -s detector -s physical_filter   butler collection-chain $REPO_SUMMIT --mode=prepend LATISS/calib LATISS/calib/DM- 33875 When that's done, I'm happy to do a quick check and review.  
            Hide
            edennihy Erik Dennihy added a comment -

            Thanks for the details, what's the appropriate path for $REPO_SUMMIT?

            Show
            edennihy Erik Dennihy added a comment - Thanks for the details, what's the appropriate path for $REPO_SUMMIT?
            Hide
            czw Christopher Waters added a comment -

            I believe at the summit it's just /repo/LATISS/

            Show
            czw Christopher Waters added a comment - I believe at the summit it's just /repo/LATISS/
            Hide
            edennihy Erik Dennihy added a comment -

            I completed the upload, ingestion, and update on the summit collection. This ticket is ready for review.

            Show
            edennihy Erik Dennihy added a comment - I completed the upload, ingestion, and update on the summit collection. This ticket is ready for review.
            Hide
            czw Christopher Waters added a comment -

            I confirmed that I was able to access both flats at the summit and NCSA.  For reference:

            import lsst.daf.butler as dafButler
             
            # butler = dafButler.Butler("/repo/LATISS", collections="LATISS/calib")
            butler = dafButler.Butler("/repo/main", collections="LATISS/calib")
             
            flatI = butler.get('flat', detector=0, exposure=2022020800095, physical_filter="SDSSi~empty")
            flatR = butler.get('flat', detector=0, exposure=2022022400091, physical_filter="SDSSr~empty")
             
            flatI.getMetadata()['CALIB_CREATE_DATE']
            '2022-03-03'
            flatR.getMetadata()['CALIB_CREATE_DATE']
            '2022-03-03'

             

            Show
            czw Christopher Waters added a comment - I confirmed that I was able to access both flats at the summit and NCSA.  For reference: import lsst.daf.butler as dafButler   # butler = dafButler.Butler( "/repo/LATISS" , collections= "LATISS/calib" ) butler = dafButler.Butler( "/repo/main" , collections= "LATISS/calib" )   flatI = butler.get( 'flat' , detector= 0 , exposure= 2022020800095 , physical_filter= "SDSSi~empty" ) flatR = butler.get( 'flat' , detector= 0 , exposure= 2022022400091 , physical_filter= "SDSSr~empty" )   flatI.getMetadata()[ 'CALIB_CREATE_DATE' ] '2022-03-03' flatR.getMetadata()[ 'CALIB_CREATE_DATE' ] '2022-03-03'  
            Hide
            edennihy Erik Dennihy added a comment -

            Marking issue as Done.

            Show
            edennihy Erik Dennihy added a comment - Marking issue as Done.

              People

              Assignee:
              edennihy Erik Dennihy
              Reporter:
              edennihy Erik Dennihy
              Reviewers:
              Christopher Waters
              Watchers:
              Christopher Waters, Erik Dennihy, Merlin Fisher-Levine
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.