Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-30696

Add task to translate gen2 fgcmcal photocalibs to gen3 photocalib catalog

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_base
    • Labels:
      None

      Description

      Add a simple task to convert the gen2 fgcm photocalib individual files to the gen3 exposure catalog format.

      Jim Bosch has offered to create the skeleton which will then be filled in. This will allow faro to be run on the gen2 to gen3 rc2 conversions.

        Attachments

        1. pipeline_pc_conv.yaml
          0.2 kB
          Eli Rykoff
        2. pipeline_wcs_conv.yaml
          0.2 kB
          Eli Rykoff

          Issue Links

          mentioned in

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            erykoff Eli Rykoff added a comment -

            Jeffrey Carlin the converted catalogs are in the collection u/erykoff/RC2/w_2021_14/DM-29519/pc_wcs. This has fgcm photocalib catalogs in gen3 format and jointcal wcs catalogs in gen3 format.

            Show
            erykoff Eli Rykoff added a comment - Jeffrey Carlin the converted catalogs are in the collection u/erykoff/RC2/w_2021_14/ DM-29519 /pc_wcs . This has fgcm photocalib catalogs in gen3 format and jointcal wcs catalogs in gen3 format.
            Hide
            Permalink
            jcarlin Jeffrey Carlin added a comment -

            Great - I ran `faro` and compared the metrics from this gen2to3 conversion and the native gen3, and they are identical for all photometry and astrometry metrics. The translation seems to be fine.

            Show
            jcarlin Jeffrey Carlin added a comment - Great - I ran `faro` and compared the metrics from this gen2to3 conversion and the native gen3, and they are identical for all photometry and astrometry metrics. The translation seems to be fine.
            Hide
            Permalink
            erykoff Eli Rykoff added a comment - - edited

            Yusra AlSayyad For the record, here are the pipetask commands that I ran for the conversion done for this ticket (one for photocalib, one for wcs):

            pipetask run -b /repo/main -i HSC/runs/RC2/w_2021_14/DM-29519 -o u/erykoff/RC2/w_2021_14/DM-29519/pc
            		 
            _wcs -p pipeline_pc_conv.yaml -d "instrument='HSC'" --register-dataset-types
            		 
            pipetask run -b /repo/main -i HSC/runs/RC2/w_2021_14/DM-29519 -o u/erykoff/RC2/w_2021_14/DM-29519/pc
            		 
            _wcs -p pipeline_wcs_conv.yaml -d "instrument='HSC'" --register-dataset-types

            And here are the pipeline yamls: pipeline_pc_conv.yaml pipeline_wcs_conv.yaml

            Show
            erykoff Eli Rykoff added a comment - - edited Yusra AlSayyad For the record, here are the pipetask commands that I ran for the conversion done for this ticket (one for photocalib, one for wcs): pipetask run -b /repo/main -i HSC/runs/RC2/w_2021_14/DM-29519 -o u/erykoff/RC2/w_2021_14/DM-29519/pc _wcs -p pipeline_pc_conv.yaml -d "instrument='HSC'" --register-dataset-types pipetask run -b /repo/main -i HSC/runs/RC2/w_2021_14/DM-29519 -o u/erykoff/RC2/w_2021_14/DM-29519/pc _wcs -p pipeline_wcs_conv.yaml -d "instrument='HSC'" --register-dataset-types And here are the pipeline yamls: pipeline_pc_conv.yaml pipeline_wcs_conv.yaml

              People

              Assignee:
              erykoff Eli Rykoff
              Reporter:
              erykoff Eli Rykoff
              Reviewers:
              Jeffrey Carlin
              Watchers:
              Eli Rykoff, Jeffrey Carlin, Jim Bosch
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.