Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31701

Enable deterministic dataset loading order in Gen3 middleware

    XMLWordPrintable

Details

    • External
    • No

    Description

      We have at least a couple of algorithms whose output is dependent on the order of the inputs (e.g jointcal & fgcm). While this may not be an issue for many processing contexts, it is critically important for our regular CI/QA reprocessing campaigns where even the smallest output differences must be scrutinized if not anticipated from explicit code changes. Thus, it is of great use to be able to ensure a deterministic ordering of inputs (visits in the above examples) within the context of the Gen3 middleware. It may be that imposing this ordering implies some efficiency hits, so it should be a configurable/optional setting.

      Attachments

        Issue Links

          Activity

            lauren Lauren MacArthur created issue -
            lauren Lauren MacArthur made changes -
            Field Original Value New Value
            Link This issue relates to DM-29821 [ DM-29821 ]
            lauren Lauren MacArthur made changes -
            Link This issue relates to DM-29820 [ DM-29820 ]
            lauren Lauren MacArthur made changes -
            Remote Link This issue links to "Page (Confluence)" [ 31004 ]
            yusra Yusra AlSayyad made changes -
            Labels drp-gen3 drp-parity drp-parity
            tjenness Tim Jenness made changes -
            Labels drp-parity drp-parity gen3-middleware
            lguy Leanne Guy made changes -
            Remote Link This issue links to "Page (Confluence)" [ 31685 ]
            jbosch Jim Bosch made changes -
            Link This issue relates to DM-33158 [ DM-33158 ]
            price Paul Price made changes -
            Assignee Paul Price [ price ]
            price Paul Price made changes -
            Team Data Release Production [ 10301 ] External [ 12117 ]
            price Paul Price made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            price Paul Price made changes -
            Reviewers Jim Bosch [ jbosch ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            jbosch Jim Bosch made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            tjenness Tim Jenness made changes -
            Component/s pipe_base [ 10727 ]
            Component/s gen3-middleware [ 19000 ]
            tjenness Tim Jenness made changes -
            Status Reviewed [ 10101 ] In Review [ 10004 ]
            tjenness Tim Jenness made changes -
            Status In Review [ 10004 ] In Progress [ 3 ]
            tjenness Tim Jenness made changes -
            Status In Progress [ 3 ] To Do [ 10001 ]
            tjenness Tim Jenness made changes -
            Resolution Done [ 10000 ]
            Status To Do [ 10001 ] Invalid [ 11005 ]

            People

              price Paul Price
              lauren Lauren MacArthur
              Jim Bosch
              Eli Rykoff, Jim Bosch, John Parejko, Lauren MacArthur, Paul Price, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.