Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10396

Run dataset for matcher validation

    XMLWordPrintable

Details

    • 6
    • Alert Production S17 - 6
    • Alert Production

    Description

      Run a large set of data through the stackified matcher. These data should be run only once and with the configuration produced by the verification process (DM-9751).

      Attachments

        Issue Links

          Activity

            I'm a little worried that we are only looking at the best behaved filters (g and r), though I know it is a bit of a tricky thing because int he harder bands ISR plays a bigger role. I think it's fine for now, but we should keep an eye out for problems in other bands.

            For completeness, I've recorded the number of calexps that got recorded for the new/old matcher. They are similar, but not the same. I hope we can figure out why these chips failed. It sounds like you have a good start on that, but we need to make sure to be able to categorize all the missing ones in the next phase.
            Completed HSC – 3952/4056
            Completed HITS – 10071/8676
            Completed CFHTLS – 8831/8906

            krughoff Simon Krughoff (Inactive) added a comment - I'm a little worried that we are only looking at the best behaved filters (g and r), though I know it is a bit of a tricky thing because int he harder bands ISR plays a bigger role. I think it's fine for now, but we should keep an eye out for problems in other bands. For completeness, I've recorded the number of calexps that got recorded for the new/old matcher. They are similar, but not the same. I hope we can figure out why these chips failed. It sounds like you have a good start on that, but we need to make sure to be able to categorize all the missing ones in the next phase. Completed HSC – 3952/4056 Completed HITS – 10071/8676 Completed CFHTLS – 8831/8906

            Ugh, I thought there would be a few u, i, z exposures in that set of visits. I'll run some specific u, i, z data as well. Should finish up today.

            As for the missing calExps, yeah, I've seen a few that had errors from the PSF estimation but there are definite failures to match to be investigated in there. The log files can point me to those and as part of DM-10398.

            cmorrison Chris Morrison [X] (Inactive) added a comment - Ugh, I thought there would be a few u, i, z exposures in that set of visits. I'll run some specific u, i, z data as well. Should finish up today. As for the missing calExps, yeah, I've seen a few that had errors from the PSF estimation but there are definite failures to match to be investigated in there. The log files can point me to those and as part of DM-10398 .

            By the time we get to happy hour there should be new 2000 ccds worth processed data for each band in /project/morriscb/CFHTLS/rerun/VALIDATE_uiz or /project/morriscb/CFHTLS/rerun/VALIDATE_stack_uiz for the new and stack matcher respectively. Let me know when you get a chance to take a look at it.

            cmorrison Chris Morrison [X] (Inactive) added a comment - By the time we get to happy hour there should be new 2000 ccds worth processed data for each band in /project/morriscb/CFHTLS/rerun/VALIDATE_uiz or /project/morriscb/CFHTLS/rerun/VALIDATE_stack_uiz for the new and stack matcher respectively. Let me know when you get a chance to take a look at it.

            O.K. I've verified there are around 2000 calexps in each band in both repositories. I think we can consider this done.

            krughoff Simon Krughoff (Inactive) added a comment - O.K. I've verified there are around 2000 calexps in each band in both repositories. I think we can consider this done.

            Completed run of data for CFHTLS, HITS (DECam) and New Horizons (HSC).

            cmorrison Chris Morrison [X] (Inactive) added a comment - Completed run of data for CFHTLS, HITS (DECam) and New Horizons (HSC).

            People

              cmorrison Chris Morrison [X] (Inactive)
              krughoff Simon Krughoff (Inactive)
              Simon Krughoff (Inactive)
              Chris Morrison [X] (Inactive), Hsin-Fang Chiang, Paul Price, Simon Krughoff (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.