Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31701

Enable deterministic dataset loading order in Gen3 middleware

    XMLWordPrintable

    Details

    • Team:
      External
    • Urgent?:
      No

      Description

      We have at least a couple of algorithms whose output is dependent on the order of the inputs (e.g jointcal & fgcm). While this may not be an issue for many processing contexts, it is critically important for our regular CI/QA reprocessing campaigns where even the smallest output differences must be scrutinized if not anticipated from explicit code changes. Thus, it is of great use to be able to ensure a deterministic ordering of inputs (visits in the above examples) within the context of the Gen3 middleware. It may be that imposing this ordering implies some efficiency hits, so it should be a configurable/optional setting.

        Attachments

          Issue Links

            Activity

            lauren Lauren MacArthur created issue -
            lauren Lauren MacArthur made changes -
            Field Original Value New Value
            Link This issue relates to DM-29821 [ DM-29821 ]
            lauren Lauren MacArthur made changes -
            Link This issue relates to DM-29820 [ DM-29820 ]
            lauren Lauren MacArthur made changes -
            Remote Link This issue links to "Page (Confluence)" [ 31004 ]
            yusra Yusra AlSayyad made changes -
            Labels drp-gen3 drp-parity drp-parity
            tjenness Tim Jenness made changes -
            Labels drp-parity drp-parity gen3-middleware
            lguy Leanne Guy made changes -
            Remote Link This issue links to "Page (Confluence)" [ 31685 ]
            jbosch Jim Bosch made changes -
            Link This issue relates to DM-33158 [ DM-33158 ]
            price Paul Price made changes -
            Assignee Paul Price [ price ]
            price Paul Price made changes -
            Team Data Release Production [ 10301 ] External [ 12117 ]
            price Paul Price made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            price Paul Price made changes -
            Reviewers Jim Bosch [ jbosch ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            jbosch Jim Bosch made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            tjenness Tim Jenness made changes -
            Component/s pipe_base [ 10727 ]
            Component/s gen3-middleware [ 19000 ]

              People

              Assignee:
              price Paul Price
              Reporter:
              lauren Lauren MacArthur
              Reviewers:
              Jim Bosch
              Watchers:
              Eli Rykoff, Jim Bosch, John Parejko, Lauren MacArthur, Paul Price, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:

                  Jenkins

                  No builds found.