Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31698

Move ConvertReferenceCatalog classes out of ingestIndex file

    XMLWordPrintable

    Details

    • Story Points:
      1
    • Urgent?:
      No

      Description

      Once gen2 has been removed, the base class and configs for the new ConvertReferenceCatalog should be moved out of ingestIndexReferenceTask.py and into convertReferenceCatalog.py. We could potentially remove the base class entirely at that point, and just have ConvertReferenceCatalog by itself; the base class was just to let gen2/3 share most of the working code. See the note at the top of ingestIndexReferenceTask.py

      Those cannot be moved until post-gen2, because the gen2 refcat config files have an explicit check that DatasetConfig live in ingestIndexReferenceTask.

        Attachments

          Issue Links

            Activity

            No builds found.
            Parejkoj John Parejko created issue -
            Parejkoj John Parejko made changes -
            Field Original Value New Value
            Link This issue is blocked by DM-20572 [ DM-20572 ]
            Parejkoj John Parejko made changes -
            Link This issue is blocked by DM-29543 [ DM-29543 ]
            Parejkoj John Parejko made changes -
            Link This issue is blocked by DM-29543 [ DM-29543 ]
            Parejkoj John Parejko made changes -
            Link This issue is triggered by DM-29543 [ DM-29543 ]
            Parejkoj John Parejko made changes -
            Description Once gen2 has been removed, the base class and configs for the new {{ConvertReferenceCatalog}} should be moved out of {{ingestIndexReferenceTask.py}} and into {{convertReferenceCatalog.py}}. We could potentially remove the base class entirely at that point, and just have {{ConvertReferenceCatalog}} by itself; the base class was just to let gen2/3 share most of the working code.

            Those cannot be moved until post-gen2, because the gen2 refcat config files have an explicit check that {{DatasetConfig}} live in {{ingestIndexReferenceTask}}.
            Once gen2 has been removed, the base class and configs for the new {{ConvertReferenceCatalog}} should be moved out of {{ingestIndexReferenceTask.py}} and into {{convertReferenceCatalog.py}}. We could potentially remove the base class entirely at that point, and just have {{ConvertReferenceCatalog}} by itself; the base class was just to let gen2/3 share most of the working code. See the note at the top of {{ingestIndexReferenceTask.py}}

            Those cannot be moved until post-gen2, because the gen2 refcat config files have an explicit check that {{DatasetConfig}} live in {{ingestIndexReferenceTask}}.
            Parejkoj John Parejko made changes -
            Link This issue relates to DM-35671 [ DM-35671 ]
            Hide
            erykoff Eli Rykoff added a comment -

            In DM-35671 I did some relevant deprecations, including LoadIndexedReferenceObjectsConfig and LoadIndexedReferenceObjectsTask. I also made it so the (still poorly named; hence this ticket) preferred LoadReferenceObjectsConfig has a cal_ref_cat config field so that we can easily move from deprecated pexConfig.ConfigurableField(target=LoadIndexedReferenceObjectsTask) to undeprecated pexConfig.ConfigField(dtype=LoadReferenceObjectsConfig) (though we need to remove any overrides in configs of the deprecated cal_ref_cat field).

            Also note that currently there are tests test_loadReferenceObjects.py and test_referenceObjectLoader.py that both test different functionality of ReferenceObjectLoader. These should be consolidated on this ticket.

            Show
            erykoff Eli Rykoff added a comment - In DM-35671 I did some relevant deprecations, including LoadIndexedReferenceObjectsConfig and LoadIndexedReferenceObjectsTask . I also made it so the (still poorly named; hence this ticket) preferred LoadReferenceObjectsConfig has a cal_ref_cat config field so that we can easily move from deprecated pexConfig.ConfigurableField(target=LoadIndexedReferenceObjectsTask) to undeprecated pexConfig.ConfigField(dtype=LoadReferenceObjectsConfig) (though we need to remove any overrides in configs of the deprecated cal_ref_cat field). Also note that currently there are tests test_loadReferenceObjects.py and test_referenceObjectLoader.py that both test different functionality of ReferenceObjectLoader . These should be consolidated on this ticket.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              Parejkoj John Parejko
              Watchers:
              Eli Rykoff, John Parejko
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:

                  Jenkins Builds

                  No builds found.