Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-820

Simplify copying tables while adding columns

    Details

    • Type: Improvement
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: afw
    • Labels:
      None
    • Story Points:
      4
    • Team:
      Data Release Production

      Description

      Currently, if I want to copy a table while adding a few columns (as specified by schema in the example) I need to do something like:

              cat = afwTable.SourceCatalog(schema)
              cat.table.defineCentroid(srcCat.table.getCentroidDefinition())
              cat.table.definePsfFlux(srcCat.table.getPsfFluxDefinition())
              # etc.
       
              scm = afwTable.SchemaMapper(srcCat.getSchema(), schema)
              for schEl in srcCat.getSchema():
                  scm.addMapping(schEl.getKey(), True)
       
              cat.extend(srcCat, True, scm)

      Please make this easier! For example

      • by adding a flag to the SchemaMapper constructor that automatically does the addMapping (should this be the default?)
      • by making it possible to copy all the slots (maybe this'll be the case when the new alias scheme is implemented?).

      Maybe we just need a new method:

      cat = srcCat.extend(schema)

      that does all the above steps.

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment - - edited

            I think it makes a lot of sense to just add a bool argument to the constructor to add the mappings. Making it default would probably be a good idea too.

            In the meantime (this is not an excuse for the current state of affairs), the way I originally intended this was that the SchemaMapper would be used to construct the output schema from the ground up, and that this would happen before the output catalog was created, so if you can afford to swap the order of things, you can avoid the explicit loop over fields:

            # Create a new SchemaMapper with an empty output Schema
            scm = afwTable.SchemaMapper(srcCat.getSchema())
             
            # Copy all fields in the given schema to the output schema, and add mappings
            # if there's a field of the same name in the input schema (which there will be
            # for every field, since they're the same)
            scm.addMinimalSchema(srcCat.getSchema(), True)
             
            # Edit the output schema to add whatever additional columns you want.
            # Note that editOutputSchema() (which is relatively new) returns a mutable
            # reference to scm's output schema, while getOutputSchema() does not.
            mutateSchema(scm.editOutputSchema())
             
            cat = afwTable.SourceCatalog(scm.getOutputSchema())
             
            cat.table.defineCentroid(srcCat.table.getCentroidDefinition())
            cat.table.definePsfFlux(srcCat.table.getPsfFluxDefinition())
            # etc.
             
            # actually copy the catalog
            cat.extend(srcCat, mapper=scm)

            Show
            jbosch Jim Bosch added a comment - - edited I think it makes a lot of sense to just add a bool argument to the constructor to add the mappings. Making it default would probably be a good idea too. In the meantime (this is not an excuse for the current state of affairs), the way I originally intended this was that the SchemaMapper would be used to construct the output schema from the ground up, and that this would happen before the output catalog was created, so if you can afford to swap the order of things, you can avoid the explicit loop over fields: # Create a new SchemaMapper with an empty output Schema scm = afwTable.SchemaMapper(srcCat.getSchema())   # Copy all fields in the given schema to the output schema, and add mappings # if there's a field of the same name in the input schema (which there will be # for every field, since they're the same) scm.addMinimalSchema(srcCat.getSchema(), True )   # Edit the output schema to add whatever additional columns you want. # Note that editOutputSchema() (which is relatively new) returns a mutable # reference to scm's output schema, while getOutputSchema() does not. mutateSchema(scm.editOutputSchema())   cat = afwTable.SourceCatalog(scm.getOutputSchema())   cat.table.defineCentroid(srcCat.table.getCentroidDefinition()) cat.table.definePsfFlux(srcCat.table.getPsfFluxDefinition()) # etc.   # actually copy the catalog cat.extend(srcCat, mapper = scm)
            Hide
            swinbank John Swinbank added a comment -

            Five years on, and with no great enthusiasm for further buy-in to afw::table, I think we can close this.

            Show
            swinbank John Swinbank added a comment - Five years on, and with no great enthusiasm for further buy-in to afw::table, I think we can close this.

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                rhl Robert Lupton
                Watchers:
                Jim Bosch, John Swinbank, Robert Lupton
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: