Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33875

Create LATISS SDSSr and SDSSi flats

    XMLWordPrintable

Details

    • Improvement
    • Status: Done
    • Resolution: Done
    • None
    • None
    • DM Science

    Description

      Create LATISS SDSSr and SDSSi flats using the data from the Feb 2022 run and add them to the LATISS/calib chained collection

      Attachments

        Issue Links

          Activity

            edennihy Erik Dennihy added a comment -

            Generated the collections LATISS/tickets/DM-33875 at NCSA, it includes SDSSr and SDSSi flats. Flats were verified through notebook visualizations. Processing from raw should only require adding LATISS/calib/DM-33875 to the list of standard input collections for AuxTel data processing.

            NOTE: The SDSSi flats are a composite of only 10 images taken on 2022-02-08. This should be replaced by a more extensive set in the future when available for higher quality flats.

            edennihy Erik Dennihy added a comment - Generated the collections LATISS/tickets/ DM-33875 at NCSA, it includes SDSSr and SDSSi flats. Flats were verified through notebook visualizations. Processing from raw should only require adding LATISS/calib/ DM-33875 to the list of standard input collections for AuxTel data processing. NOTE: The SDSSi flats are a composite of only 10 images taken on 2022-02-08. This should be replaced by a more extensive set in the future when available for higher quality flats.

            These flats look fine to me as well.  To make them available for use, you should add them to the main LATISS calib collection:

            butler collection-chain /repo/main --mode=prepend LATISS/calib LATISS/calib/DM-33875 

            Next, to synchronize the summit, you'll need to export them:

            butler export-calibs /repo/main ./calibs.dm33875 LATISS/calib/DM-33875 

            Copy that directory to the summit, and then ingest and update the summit collection as well:

            butler import --export-file ./calibs.dm33875/export.yaml --transfer copy $REPO_SUMMIT ./calibs.dm33875/ -s instrument -s detector -s physical_filter
             
            butler collection-chain $REPO_SUMMIT --mode=prepend LATISS/calib LATISS/calib/DM-33875 
            

            When that's done, I'm happy to do a quick check and review.

             

            czw Christopher Waters added a comment - These flats look fine to me as well.  To make them available for use, you should add them to the main LATISS calib collection: butler collection-chain /repo/main --mode=prepend LATISS/calib LATISS/calib/DM- 33875 Next, to synchronize the summit, you'll need to export them: butler export-calibs /repo/main ./calibs.dm33875 LATISS/calib/DM- 33875 Copy that directory to the summit, and then ingest and update the summit collection as well: butler import --export-file ./calibs.dm33875/export.yaml --transfer copy $REPO_SUMMIT ./calibs.dm33875/ -s instrument -s detector -s physical_filter   butler collection-chain $REPO_SUMMIT --mode=prepend LATISS/calib LATISS/calib/DM- 33875 When that's done, I'm happy to do a quick check and review.  

            Thanks for the details, what's the appropriate path for $REPO_SUMMIT?

            edennihy Erik Dennihy added a comment - Thanks for the details, what's the appropriate path for $REPO_SUMMIT?

            I believe at the summit it's just /repo/LATISS/

            czw Christopher Waters added a comment - I believe at the summit it's just /repo/LATISS/

            I completed the upload, ingestion, and update on the summit collection. This ticket is ready for review.

            edennihy Erik Dennihy added a comment - I completed the upload, ingestion, and update on the summit collection. This ticket is ready for review.

            I confirmed that I was able to access both flats at the summit and NCSA.  For reference:

            import lsst.daf.butler as dafButler
             
            # butler = dafButler.Butler("/repo/LATISS", collections="LATISS/calib")
            butler = dafButler.Butler("/repo/main", collections="LATISS/calib")
             
            flatI = butler.get('flat', detector=0, exposure=2022020800095, physical_filter="SDSSi~empty")
            flatR = butler.get('flat', detector=0, exposure=2022022400091, physical_filter="SDSSr~empty")
             
            flatI.getMetadata()['CALIB_CREATE_DATE']
            '2022-03-03'
            flatR.getMetadata()['CALIB_CREATE_DATE']
            '2022-03-03'

             

            czw Christopher Waters added a comment - I confirmed that I was able to access both flats at the summit and NCSA.  For reference: import lsst.daf.butler as dafButler   # butler = dafButler.Butler( "/repo/LATISS" , collections= "LATISS/calib" ) butler = dafButler.Butler( "/repo/main" , collections= "LATISS/calib" )   flatI = butler.get( 'flat' , detector= 0 , exposure= 2022020800095 , physical_filter= "SDSSi~empty" ) flatR = butler.get( 'flat' , detector= 0 , exposure= 2022022400091 , physical_filter= "SDSSr~empty" )   flatI.getMetadata()[ 'CALIB_CREATE_DATE' ] '2022-03-03' flatR.getMetadata()[ 'CALIB_CREATE_DATE' ] '2022-03-03'  
            edennihy Erik Dennihy added a comment -

            Marking issue as Done.

            edennihy Erik Dennihy added a comment - Marking issue as Done.

            People

              edennihy Erik Dennihy
              edennihy Erik Dennihy
              Christopher Waters
              Christopher Waters, Erik Dennihy, Merlin Fisher-Levine
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.