Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10398

Analyze run metadata from validation runs

    XMLWordPrintable

    Details

    • Story Points:
      8
    • Epic Link:
    • Sprint:
      Alert Production S17 - 6, Alert Production F17 - 7, AP S18-1, AP S18-2, AP S18-3
    • Team:
      Alert Production

      Description

      Analyze and make summary plots for the metadata collected in DM-12440. This may involve investigating further any failure modes that may have cropped up.

        Attachments

        1. CFHTLS_nmatch_v_nmatch.png
          CFHTLS_nmatch_v_nmatch.png
          90 kB
        2. CFHTLS_scatter_v_scatter.png
          CFHTLS_scatter_v_scatter.png
          99 kB
        3. HiTS_nmatch_v_nmatch.png
          HiTS_nmatch_v_nmatch.png
          88 kB
        4. HiTS_scatter_v_scatter.png
          HiTS_scatter_v_scatter.png
          100 kB
        5. HSC_nmatch_v_nmatch.png
          HSC_nmatch_v_nmatch.png
          99 kB
        6. HSC_scatter_v_scatter.png
          HSC_scatter_v_scatter.png
          97 kB

          Issue Links

            Activity

            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Attached summary statistics and select summary plots to this ticket showing the performance of the new MatchPessimisticB task vs the old MatchOptimsiticB task. Let me know if the summaries are sufficiently informative and suggest any other tests to be carried out.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Attached summary statistics and select summary plots to this ticket showing the performance of the new MatchPessimisticB task vs the old MatchOptimsiticB task. Let me know if the summaries are sufficiently informative and suggest any other tests to be carried out.
            Hide
            ctslater Colin Slater added a comment -

            The data here are all excellent and the resulting conclusion is very clear. I did some math to compute fractions of successful CCDs because I couldn't find those in the tables, but then discovered that the numbers were already there. So the main thing I would modify is to present this in a way that's more easily scanned to get a high level overview. JIRA is clearly not a good venue for that, so I would suggest pushing this into a technote in this ticket, even if the text surrounding it is left for a subsequent ticket. (e.g., you can make this Appendix A without anything else yet). One table of simply old percentage -> new percentage for each dataset would be the right level of detail for someone to see the conclusion, and then probably a second table could contain the detailed CCD counts and statistics.

            I think you could show that the matcher is even better speed-wise if you included a bit of data on dense fields. Doesn't have to be 10k ccds, but maybe a dozen to give a general sense. The one other minor detail is to please drop insignificant digits.

            Show
            ctslater Colin Slater added a comment - The data here are all excellent and the resulting conclusion is very clear. I did some math to compute fractions of successful CCDs because I couldn't find those in the tables, but then discovered that the numbers were already there. So the main thing I would modify is to present this in a way that's more easily scanned to get a high level overview. JIRA is clearly not a good venue for that, so I would suggest pushing this into a technote in this ticket, even if the text surrounding it is left for a subsequent ticket. (e.g., you can make this Appendix A without anything else yet). One table of simply old percentage -> new percentage for each dataset would be the right level of detail for someone to see the conclusion, and then probably a second table could contain the detailed CCD counts and statistics. I think you could show that the matcher is even better speed-wise if you included a bit of data on dense fields. Doesn't have to be 10k ccds, but maybe a dozen to give a general sense. The one other minor detail is to please drop insignificant digits.
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            I've added the summary tables to DMTN-031. They are within this commit: https://github.com/lsst-dm/dmtn-031/commit/8226d9306827b2aaa4be1dce8c7015755992bd8a

            The timing comparison using HSC is currently running and should complete some time tomorrow.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - I've added the summary tables to DMTN-031. They are within this commit: https://github.com/lsst-dm/dmtn-031/commit/8226d9306827b2aaa4be1dce8c7015755992bd8a The timing comparison using HSC is currently running and should complete some time tomorrow.
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Here are the timings for for the HSC, New Horizon pointing (Number: 908):

            New Matcher Times: Mean=86.126 Median=15.996 Std=112.800
            Old Matcher Times: Mean=68.690 Median=12.347 Std=123.853

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Here are the timings for for the HSC, New Horizon pointing (Number: 908): New Matcher Times: Mean=86.126 Median=15.996 Std=112.800 Old Matcher Times: Mean=68.690 Median=12.347 Std=123.853
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Added summary data in lsst-dm/dmtn-031.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Added summary data in lsst-dm/dmtn-031.

              People

              Assignee:
              cmorrison Chris Morrison [X] (Inactive)
              Reporter:
              krughoff Simon Krughoff
              Reviewers:
              Colin Slater
              Watchers:
              Chris Morrison [X] (Inactive), Colin Slater, John Swinbank, Simon Krughoff
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.