Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9871

Move wcs and fcr datasets out of {pointing} directory in obs_subaru

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_subaru
    • Labels:
    • Story Points:
      0.5
    • Sprint:
      DRP S17-4
    • Team:
      Data Release Production

      Description

      We should change the templates for fcr and wcs to the following:

      jointcal-results/%(tract)d/wcs-%(visit)07d-%(ccd)03d.fits
      jointcal-results/%(tract)d/fcr-%(visit)07d-%(ccd)03d.fits
      

      This will avoid problems in copying/updating directories at NAOJ during incremental releases.

        Attachments

          Issue Links

            Activity

            Hide
            price Paul Price added a comment -

            Jim Bosch, is this what you had in mind?

            price@pap-laptop:~/LSST/obs_subaru (tickets/DM-9871=) $ git sub-patch
            commit 90d0d4748a2143c88e5e656edaa58abb6bd01604
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Mon Mar 20 11:23:32 2017 -0400
             
                mappers: change wcs,fcr file templates
                
                Writing in a directory outside of the 'pointing' directory
                avoids problems in copying/updating directories at NAOJ during
                incremental releases.
             
            diff --git a/policy/HscMapper.paf b/policy/HscMapper.paf
            index 18d2071..ef13cad 100644
            --- a/policy/HscMapper.paf
            +++ b/policy/HscMapper.paf
            @@ -63,10 +63,10 @@ exposures: {
                     tables:        "raw_visit"
                 }
                 wcs: {
            -        template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/wcs-%(visit)07d-%(ccd)03d.fits"
            +        template: "jointcal-results/%(tract)d/wcs-%(visit)07d-%(ccd)03d.fits"
                 }
                 fcr: {
            -        template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/fcr-%(visit)07d-%(ccd)03d.fits"
            +        template: "jointcal-results/%(tract)d/fcr-%(visit)07d-%(ccd)03d.fits"
                     python:        "lsst.afw.image.ExposureI"
                     persistable:        "ExposureI"
                     storage:    "FitsStorage"
            diff --git a/policy/SuprimecamMapper.paf b/policy/SuprimecamMapper.paf
            index 0bee418..3f5414d 100644
            --- a/policy/SuprimecamMapper.paf
            +++ b/policy/SuprimecamMapper.paf
            @@ -58,11 +58,11 @@ exposures: {
                     tables:        "raw_visit"
                 }
                 wcs: {
            -        template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/wcs%(visit)07d%(ccd)1d.fits"
            +        template: "jointcal-results/%(tract)d/wcs-%(visit)07d-%(ccd)03d.fits"
                     columns:       "pointing" "filter" "visit" "ccd"
                 }
                 fcr: {
            -        template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/fcr%(visit)07d%(ccd)1d.fits"
            +        template: "jointcal-results/%(tract)d/fcr-%(visit)07d-%(ccd)03d.fits"
                     python:        "lsst.afw.image.ExposureI"
                     persistable:        "ExposureI"
                     storage:    "FitsStorage"
            

            Show
            price Paul Price added a comment - Jim Bosch , is this what you had in mind? price@pap-laptop:~/LSST/obs_subaru (tickets/DM-9871=) $ git sub-patch commit 90d0d4748a2143c88e5e656edaa58abb6bd01604 Author: Paul Price <price@astro.princeton.edu> Date: Mon Mar 20 11:23:32 2017 -0400   mappers: change wcs,fcr file templates Writing in a directory outside of the 'pointing' directory avoids problems in copying/updating directories at NAOJ during incremental releases.   diff --git a/policy/HscMapper.paf b/policy/HscMapper.paf index 18d2071..ef13cad 100644 --- a/policy/HscMapper.paf +++ b/policy/HscMapper.paf @@ -63,10 +63,10 @@ exposures: { tables: "raw_visit" } wcs: { - template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/wcs-%(visit)07d-%(ccd)03d.fits" + template: "jointcal-results/%(tract)d/wcs-%(visit)07d-%(ccd)03d.fits" } fcr: { - template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/fcr-%(visit)07d-%(ccd)03d.fits" + template: "jointcal-results/%(tract)d/fcr-%(visit)07d-%(ccd)03d.fits" python: "lsst.afw.image.ExposureI" persistable: "ExposureI" storage: "FitsStorage" diff --git a/policy/SuprimecamMapper.paf b/policy/SuprimecamMapper.paf index 0bee418..3f5414d 100644 --- a/policy/SuprimecamMapper.paf +++ b/policy/SuprimecamMapper.paf @@ -58,11 +58,11 @@ exposures: { tables: "raw_visit" } wcs: { - template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/wcs%(visit)07d%(ccd)1d.fits" + template: "jointcal-results/%(tract)d/wcs-%(visit)07d-%(ccd)03d.fits" columns: "pointing" "filter" "visit" "ccd" } fcr: { - template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/fcr%(visit)07d%(ccd)1d.fits" + template: "jointcal-results/%(tract)d/fcr-%(visit)07d-%(ccd)03d.fits" python: "lsst.afw.image.ExposureI" persistable: "ExposureI" storage: "FitsStorage"
            Hide
            jbosch Jim Bosch added a comment -

            Yes, but now that I look carefully at the diffs we should probably use "%(tract)04d" for that piece of the template.

            Show
            jbosch Jim Bosch added a comment - Yes, but now that I look carefully at the diffs we should probably use "%(tract)04d" for that piece of the template.
            Hide
            price Paul Price added a comment -

            Fixed:

            price@pap-laptop:~/LSST/obs_subaru (tickets/DM-9871=) $ git sub-patch
            commit 50ee17d56542b50f672603a62e163f237c4f0e6d
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Mon Mar 20 11:23:32 2017 -0400
             
                mappers: change wcs,fcr file templates
                
                Writing in a directory outside of the 'pointing' directory
                avoids problems in copying/updating directories at NAOJ during
                incremental releases.
             
            diff --git a/policy/HscMapper.paf b/policy/HscMapper.paf
            index 18d2071..5fdd5e9 100644
            --- a/policy/HscMapper.paf
            +++ b/policy/HscMapper.paf
            @@ -63,10 +63,10 @@ exposures: {
                     tables:        "raw_visit"
                 }
                 wcs: {
            -        template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/wcs-%(visit)07d-%(ccd)03d.fits"
            +        template: "jointcal-results/%(tract)04d/wcs-%(visit)07d-%(ccd)03d.fits"
                 }
                 fcr: {
            -        template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/fcr-%(visit)07d-%(ccd)03d.fits"
            +        template: "jointcal-results/%(tract)04d/fcr-%(visit)07d-%(ccd)03d.fits"
                     python:        "lsst.afw.image.ExposureI"
                     persistable:        "ExposureI"
                     storage:    "FitsStorage"
            diff --git a/policy/SuprimecamMapper.paf b/policy/SuprimecamMapper.paf
            index 0bee418..5bd29e6 100644
            --- a/policy/SuprimecamMapper.paf
            +++ b/policy/SuprimecamMapper.paf
            @@ -58,11 +58,11 @@ exposures: {
                     tables:        "raw_visit"
                 }
                 wcs: {
            -        template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/wcs%(visit)07d%(ccd)1d.fits"
            +        template: "jointcal-results/%(tract)04d/wcs-%(visit)07d-%(ccd)03d.fits"
                     columns:       "pointing" "filter" "visit" "ccd"
                 }
                 fcr: {
            -        template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/fcr%(visit)07d%(ccd)1d.fits"
            +        template: "jointcal-results/%(tract)04d/fcr-%(visit)07d-%(ccd)03d.fits"
                     python:        "lsst.afw.image.ExposureI"
                     persistable:        "ExposureI"
                     storage:    "FitsStorage"
            

            Show
            price Paul Price added a comment - Fixed: price@pap-laptop:~/LSST/obs_subaru (tickets/DM-9871=) $ git sub-patch commit 50ee17d56542b50f672603a62e163f237c4f0e6d Author: Paul Price <price@astro.princeton.edu> Date: Mon Mar 20 11:23:32 2017 -0400   mappers: change wcs,fcr file templates Writing in a directory outside of the 'pointing' directory avoids problems in copying/updating directories at NAOJ during incremental releases.   diff --git a/policy/HscMapper.paf b/policy/HscMapper.paf index 18d2071..5fdd5e9 100644 --- a/policy/HscMapper.paf +++ b/policy/HscMapper.paf @@ -63,10 +63,10 @@ exposures: { tables: "raw_visit" } wcs: { - template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/wcs-%(visit)07d-%(ccd)03d.fits" + template: "jointcal-results/%(tract)04d/wcs-%(visit)07d-%(ccd)03d.fits" } fcr: { - template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/fcr-%(visit)07d-%(ccd)03d.fits" + template: "jointcal-results/%(tract)04d/fcr-%(visit)07d-%(ccd)03d.fits" python: "lsst.afw.image.ExposureI" persistable: "ExposureI" storage: "FitsStorage" diff --git a/policy/SuprimecamMapper.paf b/policy/SuprimecamMapper.paf index 0bee418..5bd29e6 100644 --- a/policy/SuprimecamMapper.paf +++ b/policy/SuprimecamMapper.paf @@ -58,11 +58,11 @@ exposures: { tables: "raw_visit" } wcs: { - template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/wcs%(visit)07d%(ccd)1d.fits" + template: "jointcal-results/%(tract)04d/wcs-%(visit)07d-%(ccd)03d.fits" columns: "pointing" "filter" "visit" "ccd" } fcr: { - template: "%(pointing)05d/%(filter)s/corr/%(tract)04d/fcr%(visit)07d%(ccd)1d.fits" + template: "jointcal-results/%(tract)04d/fcr-%(visit)07d-%(ccd)03d.fits" python: "lsst.afw.image.ExposureI" persistable: "ExposureI" storage: "FitsStorage"
            Hide
            price Paul Price added a comment -

            Lauren MacArthur should know that this will invalidate existing meas_mosaic outputs.

            Show
            price Paul Price added a comment - Lauren MacArthur should know that this will invalidate existing meas_mosaic outputs.
            Hide
            price Paul Price added a comment -

            Lauren MacArthur approved ("I’m working on w_2017_10 since meas_mosaic hasn’t been pybind11'd").

            Merged to master.

            Show
            price Paul Price added a comment - Lauren MacArthur approved ("I’m working on w_2017_10 since meas_mosaic hasn’t been pybind11'd"). Merged to master.

              People

              Assignee:
              price Paul Price
              Reporter:
              jbosch Jim Bosch
              Reviewers:
              Jim Bosch
              Watchers:
              Jim Bosch, Paul Price
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.