Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18749

Create defects file for the AuxTel sensor

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_lsst
    • Labels:
      None

      Description

      Deliverable is the defects for the AuxTel sensor in the right format for it to be applied by ISR.

      This work follows from some work being done by Andrés Alejandro Plazas Malagón on TS8 data, so shouldn't be too hard to run that code, but the output will need putting into the right format for ISR application. See obs_subaru for details on how this is currently done (I think an ASCII file is parsed at scons-time and turned into a FITS file which is then retrieve by the butler, but not via the calib registry, but I could be wrong).

        Attachments

          Issue Links

            Activity

            Hide
            mfisherlevine Merlin Fisher-Levine added a comment -

            Regarding 4) you might want to talk to Simon Krughoff and/or Tim Jenness about the new defect stuff. I don't know if that merged or how it's used (or you could take a look and try to work it out).

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - Regarding 4) you might want to talk to Simon Krughoff and/or Tim Jenness about the new defect stuff. I don't know if that merged or how it's used (or you could take a look and try to work it out).
            Hide
            tjenness Tim Jenness added a comment -

            The defects will go in obs_lsst_data and since you are generating .ecsv files I think things look good.

            Show
            tjenness Tim Jenness added a comment - The defects will go in obs_lsst_data and since you are generating .ecsv files I think things look good.
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment -

            New files using the following command (no " --clobber-config --clobber-versions --no-versions."): 
            findDefects.py /project/shared/auxTel-ACCS/ --calib /project/shared/auxTel-ACCS/CALIB --rerun /project/plazas/auxTel_accs_data_from_sep272019_run_2019oct10 --id detector=0 --visitList=2019092700008 -c makePlots=True assertSameRun=False mode=MASTER writeAs=BOTH
             

            Defects: 

             20190927T000000.ecsv

             20190927T000000.fits

            The names of the files above follow the format of the files that are currently in 

            "obs_lsst_data/latiss/defects/rxx_s00", with the date in their name (I used  2019-09-27)

             

            PDF plots: 

            defectPlot_det0_master-dark_for-exp2019092700008_2019OCT10.pdf

            defectPlot_det0_master-flat_for-exp2019092700008_2019OCT10.pdf

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - New files using the following command (no "  --clobber-config --clobber-versions --no-versions ."):  findDefects.py /project/shared/auxTel-ACCS/ --calib /project/shared/auxTel-ACCS/CALIB --rerun /project/plazas/auxTel_accs_data_from_sep272019_run_2019oct10 --id detector=0 --visitList=2019092700008 -c makePlots=True assertSameRun=False mode=MASTER writeAs=BOTH   Defects:    20190927T000000.ecsv   20190927T000000.fits The names of the files above follow the format of the files that are currently in  "obs_lsst_data/latiss/defects/rxx_s00", with the date in their name (I used  2019-09-27)   PDF plots:  defectPlot_det0_master-dark_for-exp2019092700008_2019OCT10.pdf defectPlot_det0_master-flat_for-exp2019092700008_2019OCT10.pdf
            Hide
            krughoff Simon Krughoff added a comment -

            As Tim Jenness says, if you are producing ecsv files in the correct format, it should be simple to use them.

            You will need to ingest them using ingestDefects.py [path_to_repository] $OBS_LSST_DATA_DIR/latiss/defects --calib [path_to_calib_repository]. After you do that, you should be able to say doDefects=True if you are pointing your task to the correct calibration repository.

            Show
            krughoff Simon Krughoff added a comment - As Tim Jenness says, if you are producing ecsv files in the correct format, it should be simple to use them. You will need to ingest them using ingestDefects.py [path_to_repository] $OBS_LSST_DATA_DIR/latiss/defects --calib [path_to_calib_repository] . After you do that, you should be able to say doDefects=True if you are pointing your task to the correct calibration repository.
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment -

            lsst_distrib 19.0.0+48 w_2020_07 current setup, February 17, 2020:

            • findDefects.py /project/shared/auxTel-20200207 --rerun plazas/2020FEB17_auxtel_use_defects_false --visitList 2020012700243..2020012700247^2020012700389..2020012700420 --id detector=0 -c writeAs=BOTH

            defects (in ecvs and fits format): /project/shared/auxTel-20200207/rerun/plazas/2020FEB17_auxtel_use_defects_false/calibrations/LATISS/defects/rxx_s00/

            2020-01-27T23-52-46.112137.ecsv 2020-01-27T23-52-46.112137.fits

            • findDefects.py /project/shared/auxTel-20200207 --rerun plazas/2020FEB17_auxtel_use_defects_true --visitList 2020012700243..2020012700247^2020012700389..2020012700420 --id detector=0 -c writeAs=BOTH isrForDarks.doDefect=True isrForFlats.doDefect=True

            defects (in ecvs and fits format): /project/shared/auxTel-20200207/rerun/plazas/2020FEB17_auxtel_use_defects_true/calibrations/LATISS/defects/rxx_s00/

            2020-01-27T23-52-46.112137_do_defects_true.ecvs 2020-01-27T23-52-46.112137_do_defects_true.fits

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - lsst_distrib 19.0.0+48 w_2020_07 current setup, February 17, 2020: findDefects.py /project/shared/auxTel-20200207 --rerun plazas/2020FEB17_auxtel_use_defects_false --visitList 2020012700243..2020012700247^2020012700389..2020012700420 --id detector=0 -c writeAs=BOTH defects (in ecvs and fits format): /project/shared/auxTel-20200207/rerun/plazas/2020FEB17_auxtel_use_defects_false/calibrations/LATISS/defects/rxx_s00/ 2020-01-27T23-52-46.112137.ecsv 2020-01-27T23-52-46.112137.fits findDefects.py /project/shared/auxTel-20200207 --rerun plazas/2020FEB17_auxtel_use_defects_true --visitList 2020012700243..2020012700247^2020012700389..2020012700420 --id detector=0 -c writeAs=BOTH isrForDarks.doDefect=True isrForFlats.doDefect=True defects (in ecvs and fits format): /project/shared/auxTel-20200207/rerun/plazas/2020FEB17_auxtel_use_defects_true/calibrations/LATISS/defects/rxx_s00/ 2020-01-27T23-52-46.112137_do_defects_true.ecvs 2020-01-27T23-52-46.112137_do_defects_true.fits

              People

              • Assignee:
                plazas Andrés Alejandro Plazas Malagón
                Reporter:
                mfisherlevine Merlin Fisher-Levine
                Watchers:
                Andrés Alejandro Plazas Malagón, John Swinbank, Merlin Fisher-Levine, Robert Lupton, Simon Krughoff, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel