Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-20054

Normalize dimensions in DatasetType and config mappings up-front

    XMLWordPrintable

    Details

      Description

      Deferring normalization for these objects adds unnecessary complexity, delays error reporting, and leads to incorrect behavior in DatasetType comparisons.

      This work has already been started on DM-17023, but I'm spinning it off here to keep that from growing too large.

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            Tim Jenness, could you please review the daf_butler, obs_base, pipe_tasks, and obs_subaru changes?

            Andy Salnikov, could you please review the pipe_base and ctrl_mpexec changes?

            Most changes are in daf_butler, with a fair amount in pipe_base.  Everything else is trivial and repetitive.

            The only change visible outside daf_butler is that DatasetType can now only be constructed with normalized dimensions (i.e. a DimensionGraph) and that means pipe_base's DatasetTypeDescriptor now holds the unnormalized components of a DatasetType instead of holding a DatasetType directly.

            Show
            jbosch Jim Bosch added a comment - Tim Jenness , could you please review the daf_butler , obs_base , pipe_tasks , and obs_subaru  changes? Andy Salnikov , could you please review the pipe_base and ctrl_mpexec changes? Most changes are in daf_butler, with a fair amount in pipe_base.  Everything else is trivial and repetitive. The only change visible outside daf_butler is that DatasetType can now only be constructed with normalized dimensions (i.e. a DimensionGraph) and that means pipe_base's DatasetTypeDescriptor now holds the unnormalized components of a DatasetType instead of holding a DatasetType directly.
            Hide
            salnikov Andy Salnikov added a comment -

            I reviewed pipe_base and ctrl_mpexec, looks OK. Removing myself from reviewers.

            Show
            salnikov Andy Salnikov added a comment - I reviewed pipe_base and ctrl_mpexec, looks OK. Removing myself from reviewers.
            Hide
            tjenness Tim Jenness added a comment -

            I approve with a heavy heart. I'm somewhat disappointed at how much extra overhead is now involved in testing even the simplest thing.

            Show
            tjenness Tim Jenness added a comment - I approve with a heavy heart. I'm somewhat disappointed at how much extra overhead is now involved in testing even the simplest thing.
            Hide
            jbosch Jim Bosch added a comment -

            Merged to master.  Tim Jenness, I'll look for ways to restore some of the testing flexibility we previously had in future refactors - I think I see a path to allowing new trivial dimensions to be added as long as they don't change existing relationships, for example (though it may take me a while to get to that).

             

            Show
            jbosch Jim Bosch added a comment - Merged to master.  Tim Jenness , I'll look for ways to restore some of the testing flexibility we previously had in future refactors - I think I see a path to allowing new trivial dimensions to be added as long as they don't change existing relationships, for example (though it may take me a while to get to that).  

              People

              Assignee:
              jbosch Jim Bosch
              Reporter:
              jbosch Jim Bosch
              Reviewers:
              Tim Jenness
              Watchers:
              Andy Salnikov, Jim Bosch, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  CI Builds

                  No builds found.