Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5919

Describe our composite mappings and transformation endpoints (Frames)

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: afw
    • Story Points:
      4
    • Epic Link:
    • Sprint:
      Alert Production X16 - 5, Alert Production F16 - 6
    • Team:
      Alert Production

      Description

      To use AST in the stack, we need to be clear what our different transformations (AST:Mappings) and endpoints (AST:Frames) are going to be so we can create the chain of transformations (AST:FrameSets) that will be used throughout the stack. This applies to both images and CameraGeom. We may want to produce similar descriptions for other stack objects.

      This ticket is the high-level Frames equivalent to the mathematical Mapping description in DM-5918.

      This will help us determine how we can put our current input/output image frames into the new system.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            I'm starting to put some notes together about this.

            Show
            Parejkoj John Parejko added a comment - I'm starting to put some notes together about this.
            Hide
            Parejkoj John Parejko added a comment - - edited

            The Frames LSST will likely require to model our optical system and its distortions are listed below, in the same order as the optical path. To distinguish between frames that apply to a single CCD vs. all CCDs together, I preface each frame with the following notation:

            1 == all CCDs
            N == per-CCD

            1 Sky: The spherical sky, above the atmosphere, in the ICRS coordinate system. (ra,dec radians)
            1 Top of Atmosphere: A planar sky, above the atmosphere, with seeing distortions removed. Angle relative to optical axis of entrance pupil of telesope. (x,y radians)
            1 Pupil: Angle relative to optical axis of entrance pupil of telesope. (x,y radians)
            1 Camera: Angle relative to optical axis of entrance pupil of camera. (x,y radians)
            1 Focal Plane: 2D planar approximation to the focal plane, origin on the optical axis. LSST recommends, but does not require, that the X axis is aligned along CCD rows (NOTE: the LSST camera has X on CCD columns). Currently an affine transform from Mean Pixels, but need not be. (x,y mm)
            N Mean Pixels: Nominal position on entry surface of a detector, uniformly spaced rectangular pixels), chip distortions still in place. X-axis aligned along CCD rows (direction of serial register). (x,y unbinned pixels)
            N Actual Pixels: Actual on-chip pixel positions. The transformation from actual pixels to mean pixels involves correcting for such effects as edge rolloff and tree ring distortion. (x,y unbinned pixels)

            Separate from the above is a planar approximation frame that can be useful for comparing the effects of various other distortions.

            N Tangent Pixels: Where the light would fall on the CCD with no optical distortion. An affine transformation from Atmosphere coordinates, with the center, orientation and scale matching that CCD's Mean Pixels at the center of that CCD. Transforming from Tangent Pixels to Sky results in a local tangent plane projection of the sky on this CCD. Useful for studying the effect of optical distortion. Note that a Camera Geometry provides a similar nominal Tangent Pixel Frame which is an affine transform to the Pupil. (x,y unbinned pixels)

            It can be useful to define a 3D focal plane to specify the height of the chips above/below the 2D Focal Plane frame. We may want to replace the 2D Frame with this 3D Frame if it allows us more leverage in fitting for distortions or if it helps take care of a different type of distortion. Otherwise, we likely do not want this as part of our transform sequence, as it is more complicated but may not actually gain us anything.

            1 3D Focal: 3D positions of the detectors at the optical focal plane, origin on the optical axis, X axis aligned along CCD rows. (x,y,z mm)

            afw.CameraGeom describes our currently supported camera geometry coordinate systems, from which some of the above was derived.

            I divided Pupil into two frames, Pupil and Camera, because one could have multiple cameras in the same telescope (e.g. prototype and full LSST cameras), each introducing their own distortions while the rest of the system is unchanged. We may include the camera distortions into the Pupil->Focal mapping for simplicity and because their distortions might not be numerically separable, but I could see utility in keeping those separate, particularly if the nature of the distortions from the camera and telescope are different.

            For the ease of modeling the actual distortions that occur through the optical path, one could further separate Camera and Pupil into N Frames, one per optical element:

            1 Camera could be 4 Frames: 3 lenses and 1 filter (x,y radians)
            1 Pupil could be 3 frames: Primary, Secondary, Tertiary (x,y radians)

            Whether such fine grained separation would be useful in practice remains to be seen. We should, however, be mindful of decisions that would prevent us from implementing it in the future.

            Show
            Parejkoj John Parejko added a comment - - edited The Frames LSST will likely require to model our optical system and its distortions are listed below, in the same order as the optical path. To distinguish between frames that apply to a single CCD vs. all CCDs together, I preface each frame with the following notation: 1 == all CCDs N == per-CCD 1 Sky: The spherical sky, above the atmosphere, in the ICRS coordinate system. (ra,dec radians) 1 Top of Atmosphere: A planar sky, above the atmosphere, with seeing distortions removed. Angle relative to optical axis of entrance pupil of telesope. (x,y radians) 1 Pupil: Angle relative to optical axis of entrance pupil of telesope. (x,y radians) 1 Camera: Angle relative to optical axis of entrance pupil of camera. (x,y radians) 1 Focal Plane: 2D planar approximation to the focal plane, origin on the optical axis. LSST recommends, but does not require, that the X axis is aligned along CCD rows (NOTE: the LSST camera has X on CCD columns). Currently an affine transform from Mean Pixels, but need not be. (x,y mm) N Mean Pixels: Nominal position on entry surface of a detector, uniformly spaced rectangular pixels), chip distortions still in place. X-axis aligned along CCD rows (direction of serial register). (x,y unbinned pixels) N Actual Pixels: Actual on-chip pixel positions. The transformation from actual pixels to mean pixels involves correcting for such effects as edge rolloff and tree ring distortion. (x,y unbinned pixels) Separate from the above is a planar approximation frame that can be useful for comparing the effects of various other distortions. N Tangent Pixels: Where the light would fall on the CCD with no optical distortion. An affine transformation from Atmosphere coordinates, with the center, orientation and scale matching that CCD's Mean Pixels at the center of that CCD. Transforming from Tangent Pixels to Sky results in a local tangent plane projection of the sky on this CCD. Useful for studying the effect of optical distortion. Note that a Camera Geometry provides a similar nominal Tangent Pixel Frame which is an affine transform to the Pupil. (x,y unbinned pixels) It can be useful to define a 3D focal plane to specify the height of the chips above/below the 2D Focal Plane frame. We may want to replace the 2D Frame with this 3D Frame if it allows us more leverage in fitting for distortions or if it helps take care of a different type of distortion. Otherwise, we likely do not want this as part of our transform sequence, as it is more complicated but may not actually gain us anything. 1 3D Focal: 3D positions of the detectors at the optical focal plane, origin on the optical axis, X axis aligned along CCD rows. (x,y,z mm) afw.CameraGeom describes our currently supported camera geometry coordinate systems, from which some of the above was derived. I divided Pupil into two frames, Pupil and Camera, because one could have multiple cameras in the same telescope (e.g. prototype and full LSST cameras), each introducing their own distortions while the rest of the system is unchanged. We may include the camera distortions into the Pupil->Focal mapping for simplicity and because their distortions might not be numerically separable, but I could see utility in keeping those separate, particularly if the nature of the distortions from the camera and telescope are different. For the ease of modeling the actual distortions that occur through the optical path, one could further separate Camera and Pupil into N Frames, one per optical element: 1 Camera could be 4 Frames: 3 lenses and 1 filter (x,y radians) 1 Pupil could be 3 frames: Primary, Secondary, Tertiary (x,y radians) Whether such fine grained separation would be useful in practice remains to be seen. We should, however, be mindful of decisions that would prevent us from implementing it in the future.
            Hide
            Parejkoj John Parejko added a comment -

            Diagram showing the Frames described in the text, and the graffle file I used to make it.

            Show
            Parejkoj John Parejko added a comment - Diagram showing the Frames described in the text, and the graffle file I used to make it.
            Hide
            Parejkoj John Parejko added a comment -

            I've updated the text after our discussion, and uploaded a new version of the diagram. Please re-read the text and check if the diagram is accurate.

            Show
            Parejkoj John Parejko added a comment - I've updated the text after our discussion, and uploaded a new version of the diagram. Please re-read the text and check if the diagram is accurate.
            Hide
            Parejkoj John Parejko added a comment -

            Updating frames diagram.

            Show
            Parejkoj John Parejko added a comment - Updating frames diagram.
            Hide
            Parejkoj John Parejko added a comment -

            Hopefully final version of frames diagrams.

            Show
            Parejkoj John Parejko added a comment - Hopefully final version of frames diagrams.
            Hide
            rowen Russell Owen added a comment -

            Overall this looks fine. A few minor comments:

            • I find this confusing: "N Actual Pixels: Chip-distortion corrected positions. (x,y unbinned pixels)". This is the actual pixels, warts and all. No correction has been applied. If you want to talk about correction, perhaps it would be clearer if you said something like "the transformation from actual pixels to mean pixels is a correction for such effects as edge rolloff and tree ring distortion".
            • We recommend that Focal Plane x be aligned along CCD rows, but do not require it. We do require that Pixel x be along CCD rows (as you correctly say), due to the way we interpolate over bad columns. LSST Focal Plane x is along CCD columns and this causes endless confusion.
            • I find the order of information about 3D focal confusing. You start with "In addition..." and then the next paragraph starts "1 3D Focal". Could you combine these, or swap the order of the two paragraphs (in which case also tweak the beginning of the longer paragraph to make it clear that it is providing more detail for the previous item)?
            • You say "I have separated Camera from Pupil and Focal", but I don't see how Focal Plane comes into it. Perhaps "I divided Pupil into two frames: Pupil and Camera".
            • "Note that a Camera Geometry has a similar Frame which is an affine transform to the Pupil, since CameraGeom doesn't know about the atmosphere" I am uncomfortable with this way of talking about things. Camera Geometry provides a nominal model for the system, and as you say, in the nominal world we know nothing about the atmosphere. But this sort of consideration applies more widely. I think we will have two versions of most of these frames: nominal (as provided by camera geometry) and actual, after fitting to the data.
            Show
            rowen Russell Owen added a comment - Overall this looks fine. A few minor comments: I find this confusing: "N Actual Pixels: Chip-distortion corrected positions. (x,y unbinned pixels)". This is the actual pixels, warts and all. No correction has been applied. If you want to talk about correction, perhaps it would be clearer if you said something like "the transformation from actual pixels to mean pixels is a correction for such effects as edge rolloff and tree ring distortion". We recommend that Focal Plane x be aligned along CCD rows, but do not require it. We do require that Pixel x be along CCD rows (as you correctly say), due to the way we interpolate over bad columns. LSST Focal Plane x is along CCD columns and this causes endless confusion. I find the order of information about 3D focal confusing. You start with "In addition..." and then the next paragraph starts "1 3D Focal". Could you combine these, or swap the order of the two paragraphs (in which case also tweak the beginning of the longer paragraph to make it clear that it is providing more detail for the previous item)? You say "I have separated Camera from Pupil and Focal", but I don't see how Focal Plane comes into it. Perhaps "I divided Pupil into two frames: Pupil and Camera". "Note that a Camera Geometry has a similar Frame which is an affine transform to the Pupil, since CameraGeom doesn't know about the atmosphere" I am uncomfortable with this way of talking about things. Camera Geometry provides a nominal model for the system, and as you say, in the nominal world we know nothing about the atmosphere. But this sort of consideration applies more widely. I think we will have two versions of most of these frames: nominal (as provided by camera geometry) and actual, after fitting to the data.
            Hide
            Parejkoj John Parejko added a comment -

            After several iterations, we're now happy with these frames. We may have "nominal" (cameraGeom) and "fitted" (jointcal) versions of most of them eventually.

            Show
            Parejkoj John Parejko added a comment - After several iterations, we're now happy with these frames. We may have "nominal" (cameraGeom) and "fitted" (jointcal) versions of most of them eventually.

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                Parejkoj John Parejko
                Reviewers:
                Russell Owen
                Watchers:
                John Parejko, Krzysztof Findeisen, Russell Owen, Simon Krughoff
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel