Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10396

Run dataset for matcher validation

    XMLWordPrintable

    Details

    • Story Points:
      6
    • Epic Link:
    • Sprint:
      Alert Production S17 - 6
    • Team:
      Alert Production

      Description

      Run a large set of data through the stackified matcher. These data should be run only once and with the configuration produced by the verification process (DM-9751).

        Attachments

          Issue Links

            Activity

            Hide
            krughoff Simon Krughoff added a comment -

            I'm a little worried that we are only looking at the best behaved filters (g and r), though I know it is a bit of a tricky thing because int he harder bands ISR plays a bigger role. I think it's fine for now, but we should keep an eye out for problems in other bands.

            For completeness, I've recorded the number of calexps that got recorded for the new/old matcher. They are similar, but not the same. I hope we can figure out why these chips failed. It sounds like you have a good start on that, but we need to make sure to be able to categorize all the missing ones in the next phase.
            Completed HSC – 3952/4056
            Completed HITS – 10071/8676
            Completed CFHTLS – 8831/8906

            Show
            krughoff Simon Krughoff added a comment - I'm a little worried that we are only looking at the best behaved filters (g and r), though I know it is a bit of a tricky thing because int he harder bands ISR plays a bigger role. I think it's fine for now, but we should keep an eye out for problems in other bands. For completeness, I've recorded the number of calexps that got recorded for the new/old matcher. They are similar, but not the same. I hope we can figure out why these chips failed. It sounds like you have a good start on that, but we need to make sure to be able to categorize all the missing ones in the next phase. Completed HSC – 3952/4056 Completed HITS – 10071/8676 Completed CFHTLS – 8831/8906
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Ugh, I thought there would be a few u, i, z exposures in that set of visits. I'll run some specific u, i, z data as well. Should finish up today.

            As for the missing calExps, yeah, I've seen a few that had errors from the PSF estimation but there are definite failures to match to be investigated in there. The log files can point me to those and as part of DM-10398.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Ugh, I thought there would be a few u, i, z exposures in that set of visits. I'll run some specific u, i, z data as well. Should finish up today. As for the missing calExps, yeah, I've seen a few that had errors from the PSF estimation but there are definite failures to match to be investigated in there. The log files can point me to those and as part of DM-10398 .
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            By the time we get to happy hour there should be new 2000 ccds worth processed data for each band in /project/morriscb/CFHTLS/rerun/VALIDATE_uiz or /project/morriscb/CFHTLS/rerun/VALIDATE_stack_uiz for the new and stack matcher respectively. Let me know when you get a chance to take a look at it.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - By the time we get to happy hour there should be new 2000 ccds worth processed data for each band in /project/morriscb/CFHTLS/rerun/VALIDATE_uiz or /project/morriscb/CFHTLS/rerun/VALIDATE_stack_uiz for the new and stack matcher respectively. Let me know when you get a chance to take a look at it.
            Hide
            krughoff Simon Krughoff added a comment -

            O.K. I've verified there are around 2000 calexps in each band in both repositories. I think we can consider this done.

            Show
            krughoff Simon Krughoff added a comment - O.K. I've verified there are around 2000 calexps in each band in both repositories. I think we can consider this done.
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Completed run of data for CFHTLS, HITS (DECam) and New Horizons (HSC).

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Completed run of data for CFHTLS, HITS (DECam) and New Horizons (HSC).

              People

              Assignee:
              cmorrison Chris Morrison [X] (Inactive)
              Reporter:
              krughoff Simon Krughoff
              Reviewers:
              Simon Krughoff
              Watchers:
              Chris Morrison [X] (Inactive), Hsin-Fang Chiang, Paul Price, Simon Krughoff
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins Builds

                  No builds found.