Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-22410

Evaluate qa_explorer/matchVisits and move necessary functionality to pipe_tasks

    XMLWordPrintable

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      1
    • Epic Link:
    • Sprint:
      DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May), DRP F20-1 (June), DRP F20-2 (July), DRP F20-3 (Aug), DRP F20-5 (Oct), DRP S21a (Dec Jan), DRP S21b, DRP S22A, DRP S22B, DRP S23A, DRP S23B
    • Team:
      Data Release Production

      Description

      (1) Remove dependency of pipe_analysis on qa_explorer

      (2) Determine how the outputs of qa_explorer/matchVisits are being used. Move it or an equivalent to pipe_tasks, if necessary. 

       

        Attachments

          Issue Links

            Activity

            Hide
            tmorton Tim Morton [X] (Inactive) added a comment - - edited

            The outputs of matchVisits.py (a table indexed by coadd ID, and whose columns are ID and match_distance for each visit) are used in the QA dashboard. I created this task because at the time, as far as I knew, there were not any persisted data products containing this match information, which is necessary to compute things like repeatability metrics, etc. Perhaps by now something like this exists somewhere else? If not, it seems that it should definitely be moved to/replicated in pipe_tasks.

            Show
            tmorton Tim Morton [X] (Inactive) added a comment - - edited The outputs of matchVisits.py (a table indexed by coadd ID, and whose columns are ID and match_distance for each visit) are used in the QA dashboard. I created this task because at the time, as far as I knew, there were not any persisted data products containing this match information, which is necessary to compute things like repeatability metrics, etc. Perhaps by now something like this exists somewhere else? If not, it seems that it should definitely be moved to/replicated in pipe_tasks.
            Hide
            tmorton Tim Morton [X] (Inactive) added a comment -

            Also another thought: moving matchVisits to pipe_tasks should probably also include an evaluation of its performance, and/or update of the matching algorithm, etc. I originally wrote the task with the expectation that the matching computation itself would be eventually replaced by something more tested/robust/used elsewhere in the pipeline.

            Show
            tmorton Tim Morton [X] (Inactive) added a comment - Also another thought: moving matchVisits to pipe_tasks should probably also include an evaluation of its performance, and/or update of the matching algorithm, etc. I originally wrote the task with the expectation that the matching computation itself would be eventually replaced by something more tested/robust/used elsewhere in the pipeline.
            Hide
            tmorton Tim Morton [X] (Inactive) added a comment -

            To bookmark slack discussion with Jeffrey Carlin, the output of this task would also be useful for matchedVisitMetrics.py. This makes at least two independent use cases for matchVisits.py, suggesting it might be worth it to improve and migrate it to pipe_tasks.

            Show
            tmorton Tim Morton [X] (Inactive) added a comment - To bookmark slack discussion with Jeffrey Carlin , the output of this task would also be useful for matchedVisitMetrics.py . This makes at least two independent use cases for matchVisits.py , suggesting it might be worth it to improve and migrate it to pipe_tasks.

              People

              Assignee:
              yusra Yusra AlSayyad
              Reporter:
              yusra Yusra AlSayyad
              Watchers:
              Felipe Menanteau, Lauren MacArthur, Tim Morton [X] (Inactive), Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:

                  Jenkins

                  No builds found.