Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11784

Find all the necessary data for jointcal/meas_mosaic comparison

    Details

    • Story Points:
      1
    • Sprint:
      Alert Production F17 - 9, Alert Production F17 - 10
    • Team:
      Alert Production

      Description

      The data that was selected for the jointca/meas_mosaic acceptance test is listed on this website:

      https://confluence.lsstcorp.org/display/DM/JointCal+Acceptance+Tests

      We need to confirm that it is all available and processed with a recent enough version of the stack, including necessary reference catalogs.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            Ah, and evidently we need to also specify ccd=0..8^10..103 because ccd=9 data doesn't exist. (I'm trialling jointcal on a chunk I pulled down of 15 visits on tract=9373)

            Show
            Parejkoj John Parejko added a comment - Ah, and evidently we need to also specify ccd=0..8^10..103 because ccd=9 data doesn't exist. (I'm trialling jointcal on a chunk I pulled down of 15 visits on tract=9373 )
            Hide
            Parejkoj John Parejko added a comment -

            Butler ids and data location identified. The jointcal commands will look something like this:

            jointcal.py /datasets/hsc/repo/rerun/DM-10404/SFM -C jointcalConfig.py --output ~parejkoj/lsst/compare --no-versions --id filter=HSC-Y tract=9371 ccd=0..8^10..103
            

            We can script this to loop over the tracts listed above.

            Show
            Parejkoj John Parejko added a comment - Butler ids and data location identified. The jointcal commands will look something like this: jointcal.py /datasets/hsc/repo/rerun/DM-10404/SFM -C jointcalConfig.py --output ~parejkoj/lsst/compare --no-versions --id filter=HSC-Y tract=9371 ccd=0..8^10..103 We can script this to loop over the tracts listed above.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            In case it matters, here is the list of visit/ccd IDs of which the calexp data don't exist in the DM-10404 repo:

            https://github.com/lsst-dm/DMTR-31/blob/master/DMTR-31.tex#L334-L358
            (Same list as on the confluence page)

            Show
            hchiang2 Hsin-Fang Chiang added a comment - In case it matters, here is the list of visit/ccd IDs of which the calexp data don't exist in the DM-10404 repo: https://github.com/lsst-dm/DMTR-31/blob/master/DMTR-31.tex#L334-L358 (Same list as on the confluence page)
            Hide
            lauren Lauren MacArthur added a comment - - edited

            UDEEP/HSC-G
            --id ccd=0..8^10..103 visit=1886^11688^11690^11692^11694^11696^11698^11700^11702^11704^11706^11708^11710^11712^29324^29326^29336^29340^29350 tract=9813 
            UDEEP/HSC-R
            --id ccd=0..8^10..103 visit=1200^1202^1204^1206^1208^1210^1212^1214^1216^1218^1220^1222^23690^23692^23694^23704^23706^23716^23718 tract=9813
            UDEEP/HSC-I
            --id ccd=0..8^10..103 visit=1226^1228^1230^1232^1238^1240^1242^1244^1246^1248^19656^19658^19660^19662^19680^19682^19684^19694^19696^19698^19708^19710^19712^30482^30484^30486^30488^30490^30492^30494^30496^30498^30500^30502^30504 tract=9813
            UDEEP/HSC-Z
            --id ccd=0..8^10..103 visit=1164^1166^1168^1170^1172^1174^1176^1178^1180^1182^1184^1186^1188^1190^1192^1194^17898^17900^17902^17904^17906^17908^17926^17928^17930^17932^17934^17944^17946^17948^17950^17952^17962 tract=9813
            UDEEP/HSC-Y
            --id ccd=0..8^10..103 visit=318^322^324^326^328^330^332^340^344^346^348^350^352^354^356^358^360^362^1856^1868^1870^1872^1874^1876^1880^1882^11716^11718^11720^11722^11724^11726^11728^11730^11732^11734^11736^11738^11740^22602^22604^22606^22608^22626^22628^22630^22632^22642^22644^22646^22648^22658^22660^22662^22664 tract=9813
            UDEEP/NB0921
            --id ccd=0..8^10..103 visit=23036^23038^23040^23042^23044^23046^23048^23050^23052^23054^23056^23594^23596^23598^23600^23602^23604^23606^24298^24300^24302^24304^24306^24308^24310^25810^25812^25814^25816 tract=9813
            

            Show
            lauren Lauren MacArthur added a comment - - edited UDEEP/HSC-G --id ccd=0..8^10..103 visit=1886^11688^11690^11692^11694^11696^11698^11700^11702^11704^11706^11708^11710^11712^29324^29326^29336^29340^29350 tract=9813 UDEEP/HSC-R --id ccd=0..8^10..103 visit=1200^1202^1204^1206^1208^1210^1212^1214^1216^1218^1220^1222^23690^23692^23694^23704^23706^23716^23718 tract=9813 UDEEP/HSC-I --id ccd=0..8^10..103 visit=1226^1228^1230^1232^1238^1240^1242^1244^1246^1248^19656^19658^19660^19662^19680^19682^19684^19694^19696^19698^19708^19710^19712^30482^30484^30486^30488^30490^30492^30494^30496^30498^30500^30502^30504 tract=9813 UDEEP/HSC-Z --id ccd=0..8^10..103 visit=1164^1166^1168^1170^1172^1174^1176^1178^1180^1182^1184^1186^1188^1190^1192^1194^17898^17900^17902^17904^17906^17908^17926^17928^17930^17932^17934^17944^17946^17948^17950^17952^17962 tract=9813 UDEEP/HSC-Y --id ccd=0..8^10..103 visit=318^322^324^326^328^330^332^340^344^346^348^350^352^354^356^358^360^362^1856^1868^1870^1872^1874^1876^1880^1882^11716^11718^11720^11722^11724^11726^11728^11730^11732^11734^11736^11738^11740^22602^22604^22606^22608^22626^22628^22630^22632^22642^22644^22646^22648^22658^22660^22662^22664 tract=9813 UDEEP/NB0921 --id ccd=0..8^10..103 visit=23036^23038^23040^23042^23044^23046^23048^23050^23052^23054^23056^23594^23596^23598^23600^23602^23604^23606^24298^24300^24302^24304^24306^24308^24310^25810^25812^25814^25816 tract=9813
            Hide
            lauren Lauren MacArthur added a comment -

            If the above is useful, let me know and I'll do the rest.

            Show
            lauren Lauren MacArthur added a comment - If the above is useful, let me know and I'll do the rest.

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                Parejkoj John Parejko
                Watchers:
                Hsin-Fang Chiang, Jim Bosch, John Parejko, John Swinbank, Lauren MacArthur
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel