Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-245

Implement HSC camera in new camera framework

    XMLWordPrintable

    Details

    • Story Points:
      10
    • Epic Link:
    • Sprint:
      Science Pipelines DM-W15-2, Science Pipelines DM-W15-3, Science Pipelines DM-W15-4, Science Pipelines DM-W15-5
    • Team:
      Data Release Production

      Description

      The HSC camera needs to be put in the new camera geometry. I will implement is like we did lsstSim and sdss. That is I will generate a repository with the camera config and ampInfo FITS files that can be unpersisted by the butler. These changes should only require modifying obs_subaru.

      My plan is to just use the policy file to populate the new geometry, but if there is more up to date information, I'll happily use that.

        Attachments

        1. hsc_83_89.jpg
          hsc_83_89.jpg
          58 kB
        2. hsc_fp.png
          hsc_fp.png
          74 kB
        3. sc_fp.png
          sc_fp.png
          151 kB

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            Moved this to the current sprint, and boosted the story point estimate for two reasons:

            • new person working on it who is less familiar with the codebase
            • may also need to make some non-cameraGeom modifications to obs_subaru to get it working again with the LSST pipeline
            Show
            jbosch Jim Bosch added a comment - Moved this to the current sprint, and boosted the story point estimate for two reasons: new person working on it who is less familiar with the codebase may also need to make some non-cameraGeom modifications to obs_subaru to get it working again with the LSST pipeline
            Hide
            lauren Lauren MacArthur added a comment -

            Continuing from where Simon Krughoff left off, I have updated obs_subaru such that processCcd.py will run through past the isr stage (several ccds were tested and all made it at least to the calibrate.astrometry stage).

            I have pulled over as many commits from the HSC side as seemed reasonable at this stage. In particular, work needs to be done on the measurement algorithms (currently disabled for the most part). I also have not yet pulled over any of RHL's commits related to bin/plotFocus.py and bin/showDither.py. I will look into these as part of a future ticket.

            There are many (~85) commits since tickets/DM-245 branched from master included. I have pushed these to branch u/lauren/DM-245, the last one in the log is:

            Last of Commits

            de069f3 2014-11-06 (HEAD, u/lauren/DM-245) config: set priorityList for det and meas merges [Lauren MacArthur]

            The first few are from Simon Krughoff (not yet merged to master). The first commit by me was 8ceffc2, as shown below:

            First of Commits

            8ceffc2 2014-11-19 config: remove all root.measurement.algorithms references outside try statements in processCcd.py and hsc/processCcd.py [Lauren MacArthur]
            aee9f22 2014-11-05 Removes white spaces from hscMapper.py [Simon Krughoff]
            bbe793b 2014-11-05 Adds comment about SUBARU_DATA [Simon Krughoff]
            4db5894 2014-11-05 Adds a new utility to display cameras [Simon Krughoff]
            2b62835 2014-11-04 Adds camera repository files [Simon Krughoff]
            d155bbf 2014-11-04 Adds a utility for building camera registries [Simon Krughoff]
            d0ca401 2014-11-04 Wraps distortion code in XYTransform [Simon Krughoff]
            cb4a8e2 2014-11-05 Updates genDefectFits.py to use new CameraGeom [Simon Krughoff]
            fc0a804 2014-11-05 Update Exceptions in Crosstalk to new names. [Simon Krughoff]

            The only other package that was modified and whose setup is required for this obs_subaru to run is ip_isr. This is from branch u/price/hsc-merge-20141120.

            Show
            lauren Lauren MacArthur added a comment - Continuing from where Simon Krughoff left off, I have updated obs_subaru such that processCcd.py will run through past the isr stage (several ccds were tested and all made it at least to the calibrate.astrometry stage). I have pulled over as many commits from the HSC side as seemed reasonable at this stage. In particular, work needs to be done on the measurement algorithms (currently disabled for the most part). I also have not yet pulled over any of RHL's commits related to bin/plotFocus.py and bin/showDither.py. I will look into these as part of a future ticket. There are many (~85) commits since tickets/ DM-245 branched from master included. I have pushed these to branch u/lauren/ DM-245 , the last one in the log is: Last of Commits de069f3 2014-11-06 (HEAD, u/lauren/ DM-245 ) config: set priorityList for det and meas merges [Lauren MacArthur] The first few are from Simon Krughoff (not yet merged to master). The first commit by me was 8ceffc2, as shown below: First of Commits 8ceffc2 2014-11-19 config: remove all root.measurement.algorithms references outside try statements in processCcd.py and hsc/processCcd.py [Lauren MacArthur] aee9f22 2014-11-05 Removes white spaces from hscMapper.py [Simon Krughoff] bbe793b 2014-11-05 Adds comment about SUBARU_DATA [Simon Krughoff] 4db5894 2014-11-05 Adds a new utility to display cameras [Simon Krughoff] 2b62835 2014-11-04 Adds camera repository files [Simon Krughoff] d155bbf 2014-11-04 Adds a utility for building camera registries [Simon Krughoff] d0ca401 2014-11-04 Wraps distortion code in XYTransform [Simon Krughoff] cb4a8e2 2014-11-05 Updates genDefectFits.py to use new CameraGeom [Simon Krughoff] fc0a804 2014-11-05 Update Exceptions in Crosstalk to new names. [Simon Krughoff] The only other package that was modified and whose setup is required for this obs_subaru to run is ip_isr. This is from branch u/price/hsc-merge-20141120.
            Hide
            swinbank John Swinbank added a comment -

            Folks,

            I would like to switch the "epic link" for this story to DM-1382 ("work on ideas for the deblender"). We originally defined Lauren's work under that epic as DM-1474 before realising that duplicated this story.

            This is basically just a book-keeping change, with two benefits:

            • The original parent epic of this story, DM-753, was scheduled for completion in Summer 2014. This is the only outstanding story blocking that epic from being closed.
            • The work currently being invested in this story can be counted against value earned in W15.

            If anybody find this change objectionable, please let me know.

            Show
            swinbank John Swinbank added a comment - Folks, I would like to switch the "epic link" for this story to DM-1382 ("work on ideas for the deblender"). We originally defined Lauren's work under that epic as DM-1474 before realising that duplicated this story. This is basically just a book-keeping change, with two benefits: The original parent epic of this story, DM-753 , was scheduled for completion in Summer 2014. This is the only outstanding story blocking that epic from being closed. The work currently being invested in this story can be counted against value earned in W15. If anybody find this change objectionable, please let me know.
            Hide
            price Paul Price added a comment -

            Review was done on github: https://github.com/LSST/obs_subaru/pull/2

            Lauren and I are going to meet soon to discuss how to clean some things up.

            Show
            price Paul Price added a comment - Review was done on github: https://github.com/LSST/obs_subaru/pull/2 Lauren and I are going to meet soon to discuss how to clean some things up.
            Hide
            price Paul Price added a comment -

            Here's the submission from ip_isr, on tickets/DM-245 at github.

            price@price-laptop:~/LSST/ip/isr (tickets/DM-245=) $ git --no-pager log --stat --reverse origin/master..
            commit 33c191f382ffacc82dba5250b4f5f7f78a1af28d
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Wed Jul 10 09:32:29 2013 -0400
             
                fringe: better error message for fringe unpersistence
                
                Lack of a fringe frame is often characterised by an obscure error
                from the depths of the butler that doens't mention anything about
                a fringe frame.  This provides a useful error message.
             
             python/lsst/ip/isr/fringe.py | 5 ++++-
             tests/testFringes.py         | 3 ++-
             2 files changed, 6 insertions(+), 2 deletions(-)
             
            commit 55cf411aa20e1fd45bb857e53c50592f63e1f561
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Wed Dec 4 15:50:33 2013 -0500
             
                overscan: allow masking of overscan
                
                On HSC, saturated pixels can leak into the overscan.  We
                mask them, but they need to be ignored.
                
                Added a statControl keyword argument for control of the
                rejection and masking.
             
             python/lsst/ip/isr/isr.py | 21 +++++++++++++++------
             1 file changed, 15 insertions(+), 6 deletions(-)
             
            commit 72841624d121205faa3e8391079d2b07a2272e05
            Author: Jim Bosch <jbosch@astro.princeton.edu>
            Date:   Thu Mar 13 10:18:35 2014 -1000
             
                Pass NumPy arrays to afwMath.makeInterpolate.
             
             python/lsst/ip/isr/isr.py | 4 ++--
             1 file changed, 2 insertions(+), 2 deletions(-)
             
            commit 460b03e4c40491df5efe5aa3326c980a6fbc1ec4
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Tue Apr 1 17:50:21 2014 +0000
             
                transposeMaskedImage: fix transposition
                
                Wasn't working due to numpy float vs float32 issues with ImageF.
             
             python/lsst/ip/isr/isr.py | 9 +++++----
             1 file changed, 5 insertions(+), 4 deletions(-)
             
            commit c250aaceccdb95a5edabefbb2c84d8ba61596fdf
            Author: Robert Lupton the Good <rhl@astro.princeton.edu>
            Date:   Tue Apr 8 16:11:48 2014 -0400
             
                overscan: Toughen code to handle the case that no rows are masked
                
                See https://jira.lsstcorp.org/browse/DM-415.  In this case all the rows
                were bad, but the masking code passed them all so this bug triggered
             
             python/lsst/ip/isr/isr.py | 15 ++++++++++++++-
             1 file changed, 14 insertions(+), 1 deletion(-)
             
            commit cb2504fe4f4846f34aae8f90dfb617d7398bedff
            Author: Jim Bosch <jbosch@astro.princeton.edu>
            Date:   Fri May 16 18:27:17 2014 -0400
             
                Make random number generation in fringe code deterministic
             
             python/lsst/ip/isr/fringe.py | 13 +++++++++----
             tests/testFringes.py         | 13 ++++++-------
             2 files changed, 15 insertions(+), 11 deletions(-)
             
            commit 9749e104108b0a391c913d30501489ed65503750
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Tue Oct 7 14:55:11 2014 -0400
             
                saturationDetection: apply to overscan as well as image
                
                The overscan may be saturated from charge bleeding in from
                the image.
             
             python/lsst/ip/isr/isrTask.py | 2 +-
             1 file changed, 1 insertion(+), 1 deletion(-)
             
            commit 7b465785a8142b8693ca0eae5e4ab5f5392d8df1
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Mon Oct 6 19:39:29 2014 -0400
             
                defects: activate useFallbackValueAtEdge for interpolation
                
                Lack of use of this feature meant that NANs were not being
                interpolated over.
             
             python/lsst/ip/isr/isr.py | 2 +-
             1 file changed, 1 insertion(+), 1 deletion(-)

            Show
            price Paul Price added a comment - Here's the submission from ip_isr, on tickets/ DM-245 at github. price@price-laptop:~/LSST/ip/isr (tickets/DM-245=) $ git --no-pager log --stat --reverse origin/master.. commit 33c191f382ffacc82dba5250b4f5f7f78a1af28d Author: Paul Price <price@astro.princeton.edu> Date: Wed Jul 10 09:32:29 2013 -0400   fringe: better error message for fringe unpersistence Lack of a fringe frame is often characterised by an obscure error from the depths of the butler that doens't mention anything about a fringe frame. This provides a useful error message.   python/lsst/ip/isr/fringe.py | 5 ++++- tests/testFringes.py | 3 ++- 2 files changed, 6 insertions(+), 2 deletions(-)   commit 55cf411aa20e1fd45bb857e53c50592f63e1f561 Author: Paul Price <price@astro.princeton.edu> Date: Wed Dec 4 15:50:33 2013 -0500   overscan: allow masking of overscan On HSC, saturated pixels can leak into the overscan. We mask them, but they need to be ignored. Added a statControl keyword argument for control of the rejection and masking.   python/lsst/ip/isr/isr.py | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-)   commit 72841624d121205faa3e8391079d2b07a2272e05 Author: Jim Bosch <jbosch@astro.princeton.edu> Date: Thu Mar 13 10:18:35 2014 -1000   Pass NumPy arrays to afwMath.makeInterpolate.   python/lsst/ip/isr/isr.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)   commit 460b03e4c40491df5efe5aa3326c980a6fbc1ec4 Author: Paul Price <price@astro.princeton.edu> Date: Tue Apr 1 17:50:21 2014 +0000   transposeMaskedImage: fix transposition Wasn't working due to numpy float vs float32 issues with ImageF.   python/lsst/ip/isr/isr.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)   commit c250aaceccdb95a5edabefbb2c84d8ba61596fdf Author: Robert Lupton the Good <rhl@astro.princeton.edu> Date: Tue Apr 8 16:11:48 2014 -0400   overscan: Toughen code to handle the case that no rows are masked See https://jira.lsstcorp.org/browse/DM-415. In this case all the rows were bad, but the masking code passed them all so this bug triggered   python/lsst/ip/isr/isr.py | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)   commit cb2504fe4f4846f34aae8f90dfb617d7398bedff Author: Jim Bosch <jbosch@astro.princeton.edu> Date: Fri May 16 18:27:17 2014 -0400   Make random number generation in fringe code deterministic   python/lsst/ip/isr/fringe.py | 13 +++++++++---- tests/testFringes.py | 13 ++++++------- 2 files changed, 15 insertions(+), 11 deletions(-)   commit 9749e104108b0a391c913d30501489ed65503750 Author: Paul Price <price@astro.princeton.edu> Date: Tue Oct 7 14:55:11 2014 -0400   saturationDetection: apply to overscan as well as image The overscan may be saturated from charge bleeding in from the image.   python/lsst/ip/isr/isrTask.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)   commit 7b465785a8142b8693ca0eae5e4ab5f5392d8df1 Author: Paul Price <price@astro.princeton.edu> Date: Mon Oct 6 19:39:29 2014 -0400   defects: activate useFallbackValueAtEdge for interpolation Lack of use of this feature meant that NANs were not being interpolated over.   python/lsst/ip/isr/isr.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

              People

              Assignee:
              lauren Lauren MacArthur
              Reporter:
              krughoff Simon Krughoff
              Reviewers:
              Paul Price
              Watchers:
              Jim Bosch, John Swinbank, Lauren MacArthur, Paul Price, Simon Krughoff, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins Builds

                  No builds found.