Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29732

Allow butler to retrieve single amplifier images from (at least) biases

    XMLWordPrintable

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: obs_base, obs_lsst
    • Labels:
      None
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      For most instruments this will be easy, because even single-amplifier raws are never flipped or shifted relative to the master calibrations. For LSST, we'll need to essentially invert the flipping and bbox manipulations that happen when we read raws.

      I expect DM-29370 to set up the scaffolding for this - basically a method that raises NotImplementedError to be replaced by real logic on this ticket.

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            Robert Lupton, I've discovered a potential problem with this approach that I'd like to resolve before diving into DM-29370:

            If you're writing a bit of code that's trying to load and then flip an single-amplifier subimage of a bias to match a single-amplifier subimage of a raw, how do we tell it what kinds of flips are needed?  They will be needed for LSST images (because the raw subimages will be the original, readout-coordinate ones), but not any other camera we support (where, as I understand it, the natural single-amp raw subimage is never flipped relative to the full raw).  But hard-coding

            if dataId["instrument"].startswith("LSSTCam"): <do-some-flips>

            or similar would pretty evil.

            Best idea I've had so far is to ask the user to pass in the Amplifier or Detector object (obtained from the raw) they want you to match, and this code would inspect that and say, "well, this is not the detector I have, but if I flip X but not Y, they're the same".  But that's an unpleasant kind of comparison to try to do.  What's worse is that I'm pretty sure bias is just a MaskedImage, so "detector I have" is not really a thing anyway.

            Show
            jbosch Jim Bosch added a comment - Robert Lupton , I've discovered a potential problem with this approach that I'd like to resolve before diving into DM-29370 : If you're writing a bit of code that's trying to load and then flip an single-amplifier subimage of a bias to match a single-amplifier subimage of a raw, how do we tell it what kinds of flips are needed?  They will be needed for LSST images (because the raw subimages will be the original, readout-coordinate ones), but not any other camera we support (where, as I understand it, the natural single-amp raw subimage is never flipped relative to the full raw).  But hard-coding if dataId ["instrument"] .startswith("LSSTCam"): <do-some-flips> or similar would pretty evil. Best idea I've had so far is to ask the user to pass in the Amplifier or Detector object (obtained from the raw) they want you to match, and this code would inspect that and say, "well, this is not the detector I have, but if I flip X but not Y, they're the same".  But that's an unpleasant kind of comparison to try to do.  What's worse is that I'm pretty sure bias is just a MaskedImage , so "detector I have" is not really a thing anyway.
            Hide
            jbosch Jim Bosch added a comment -

            Robert Lupton, I had a chat with Eric Charles , and the question of where to put your code is a lot simpler than I thought: he can always read a full, assembled bias (or similar) up front, because there's one (e.g.) bias per run, and then extract subimages from that and flip them in memory.  So nothing from this ticket needs to go behind a Butler API, it's not blocked at all by my DM-29370 ticket, and you can just write something with a signature like this anywhere you'd:

            def disassembleAmplifier(image: Exposure, amplifier: Amplifier) -> Exposure:
                """Extract a single-amplifier image in readout coordinates.
             
                Parameters
                ----------
                image : `Exposure`
                    Full-detector image to extract the amplifier subimage from.  This
                    can be expected to have a Detector that represents its current
                    state (which may be trimmed or untrimmed, but is definitely in
                    physical orientation).
                amplifier : `Amplifier`
                    A pre-assembly amplifier object whose subimage should be returned.
             
                Returns
                -------
                ampImage : `Exposure`
                    Exposure corresponding to a single amplifier, with a single-amplifier
                    Detector (containing exactly the given amplifier) attached.  Always
                    in readout coordinates.
                """
            

            A method of AssembleCcdTask seems like a good place to put it to me, but you may know of a better place.

            One possible complication is that I don't know if we attach Detector objects to our master calibrations; if we don't, then we can either change that or find some way to get a post-assembly full Detector to Eric's code so he can attach it (note that we probably can't get one easily from the Camera object, as IIRC that has pre-assembly Detectors).

            Show
            jbosch Jim Bosch added a comment - Robert Lupton , I had a chat with Eric Charles , and the question of where to put your code is a lot simpler than I thought: he can always read a full, assembled bias (or similar) up front, because there's one (e.g.) bias per run, and then extract subimages from that and flip them in memory.  So nothing from this ticket needs to go behind a Butler API, it's not blocked at all by my DM-29370 ticket, and you can just write something with a signature like this anywhere you'd: def disassembleAmplifier(image: Exposure, amplifier: Amplifier) -> Exposure: "" "Extract a single-amplifier image in readout coordinates.   Parameters ---------- image : `Exposure` Full-detector image to extract the amplifier subimage from. This can be expected to have a Detector that represents its current state (which may be trimmed or untrimmed, but is definitely in physical orientation). amplifier : `Amplifier` A pre-assembly amplifier object whose subimage should be returned.   Returns ------- ampImage : `Exposure` Exposure corresponding to a single amplifier, with a single-amplifier Detector (containing exactly the given amplifier) attached. Always in readout coordinates. "" " A method of AssembleCcdTask seems like a good place to put it to me, but you may know of a better place. One possible complication is that I don't know if we attach Detector objects to our master calibrations; if we don't, then we can either change that or find some way to get a post-assembly full Detector to Eric's code so he can attach it (note that we probably can't get one easily from the Camera object, as IIRC that has pre-assembly Detectors ).
            Hide
            jbosch Jim Bosch added a comment -

            I think everything that needed to be done on this ticket has been done on DM-29370, but I'll hold off on closing this one until that one is reviewed in case something unexpected pops up.

            Show
            jbosch Jim Bosch added a comment - I think everything that needed to be done on this ticket has been done on DM-29370 , but I'll hold off on closing this one until that one is reviewed in case something unexpected pops up.

              People

              Assignee:
              rhl Robert Lupton
              Reporter:
              jbosch Jim Bosch
              Watchers:
              Jim Bosch
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:

                  Jenkins

                  No builds found.