There seems a problem with the tar.gz file?

At a first glance through the document, it does look like we need to implement TPV correctly to process these. Note from the earlier ticket: "TPV is not (yet) supported by LSST software so TPV is then converted back to --TAN."

I was just going to say that, but it seems the fallback to TAN isn't happening, otherwise it wouldn't choke on the header.

Hsin-Fang Chiang, Try the tarball again. (The first one got corrupted when I uploaded it)

For now u/yusra/DM-3196 reverts the commit in DM-2967: af906a8f28afeeb7b665bb08bd543c281e718f2c

Has anyone analysed these DECAM headers for legality? And the headers post the DM-2967 change?

Are you asking what changes DM-2967 makes to the headers?

My understanding is that these are TPV headers. The new code is meant to spot the TPV-ness, change the projection to TAN and remove all except the first four PV headers. The header is then processed as a TAN header (so isn't technically correct). Something is going wrong though when the newly modified header is sent to wcsLib.

Robert Lupton might be asking for a dump of the headers after AFW has hacked them all up to turn them into TAN.

Here is an example of the header after afw changed it into TAN. The WCS projection type of the input file is TPV.

XTENSION= 'IMAGE   '           / IMAGE extension

BITPIX  =                  -32 / number of bits per data pixel

NAXIS   =                    2 / number of data axes

NAXIS1  =                 2046 / length of data axis 1

NAXIS2  =                 4094 / length of data axis 2

PCOUNT  =                    0 / required keyword; must = 0

GCOUNT  =                    1 / required keyword; must = 1

EQUINOX =      9.87654321E+107 / Equinox of coordinates

RADESYS = 'ICRS    '           / Coordinate system for equinox

CRPIX1  =               8914.4 / WCS Coordinate reference pixel

CRPIX2  =                2048. / WCS Coordinate reference pixel

CD1_1   =  -2.863583082783E-07 / WCS Coordinate scale matrix

CD1_2   =   7.285161023394E-05 / WCS Coordinate scale matrix

CD2_1   =  -7.285152832571E-05 / WCS Coordinate scale matrix

CD2_2   =  -2.881919867864E-07 / WCS Coordinate scale matrix

CRVAL1  =       300.6567689437 / WCS Ref value (RA in decimal degrees)

CRVAL2  =      -57.45059962179 / WCS Ref value (DEC in decimal degrees)

CUNIT1  = 'deg     '

CUNIT2  = 'deg     '

CTYPE1  = 'RA---TAN'           / WCS Coordinate type

CTYPE2  = 'DEC--TAN'           / WCS Coordinate type

LTV1    =                    0

LTV2    =                    0

INHERIT =                    T

EXTTYPE = 'IMAGE   '

CRVAL1A =                    0 / Column pixel of Reference Pixel

CRVAL2A =                    0 / Row pixel of Reference Pixel

CRPIX1A =                    1 / Column Pixel Coordinate of Reference

CRPIX2A =                    1 / Row Pixel Coordinate of Reference

CTYPE1A = 'LINEAR  '           / Type of projection

CTYPE2A = 'LINEAR  '           / Type of projection

CUNIT1A = 'PIXEL   '           / Column unit

CUNIT2A = 'PIXEL   '           / Row unit

There aren't any PV headers at all there. Just CD. Did all the PV headers get stripped?

The EQUINOX header is really fun. Good thing that it's ignored for ICRS.

I used u/yusra/DM-3196, otherwise makeWcs() failed as in the description. This was what I did:

>>> md = afwImage.readMetadata('file.fits.fz[10]')

>>> wcs = afwImage.makeWcs(md)

makeWcs WARNING: Stripping PVi_j keys from projection RA---TPV/DEC--TPV

>>> wcs.getFitsMetadata().toList()

[('NAXIS', 2, 'number of data axes'), ('EQUINOX', 2000.0, 'Equinox of coordinates'), ('RADESYS', 'ICRS', 'Coordinate system for equinox'), ('CRPIX1', 8914.4, 'WCS Coordinate reference pixel'), ('CRPIX2', 2048.0, 'WCS Coordinate reference pixel'), ('CD1_1', -2.863583082783e-07, 'WCS Coordinate scale matrix'), ('CD1_2', 7.285161023394e-05, 'WCS Coordinate scale matrix'), ('CD2_1', -7.285152832571e-05, 'WCS Coordinate scale matrix'), ('CD2_2', -2.881919867864e-07, 'WCS Coordinate scale matrix'), ('CRVAL1', 300.6567689437, 'WCS Ref value (RA in decimal degrees)'), ('CRVAL2', -57.45059962179, 'WCS Ref value (DEC in decimal degrees)'), ('CUNIT1', 'deg', ''), ('CUNIT2', 'deg', ''), ('CTYPE1', 'RA---TAN', 'WCS Coordinate type'), ('CTYPE2', 'DEC--TAN', 'WCS Coordinate type')]

In the header of the input file, there were PV keywords of PVi_j, i=1-2, j=0-10

I tried two ways to read the header and makeWcs: (1) the quick route as in my last comment, I was thinking it might be an easier way to debug , and (2) retrieving the image using obs_decam, and writing the result into a fits file using ExposureF.writeFits(). In the header of this output fits file, HDU 0 has CTYPE1 = 'RA---TPV' and all PVi_j cards. The header I posted earlier in https://jira.lsstcorp.org/browse/DM-3196?focusedCommentId=37871&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-37871

is from HDU 1 and it does have a funny EQUINOX header. Don't know where it got messed up.

David Nidever [X] has asked me what the DRP/Princeton team can do to help with this issue. Scanning the discussion above, I'm confused as to what's required – is this a request for full support for the TPV WCS in the LSST stack, or simply a bug report that the fallback to TAN isn't working as advertised?

Ideally we would support TPV. In the interim this bug is blocking obs_decam work because the code is documented to convert TPV to TAN and that fallback does not seem to be working. Yusra AlSayyad works around that by simply reverting the commit that broke it (see DM-2967) and usign an AFW branch, but AFW master will not work (which prevents obs_decam from being integrated into the LSST CI).

Supporting TPV natively is a whole other discussion not related to this ticket.

Moving forward on this ticket requires getting a hold of the scamp output images that Robert Lupton used to test DM-2883 and editing the commit in DM-2967 that broke the fallback so that it works on both the scamp images and decam images.

Agreed. I'll find the file and attach it here later today in some timezone – have to dash now.

I pushed a fix to tickets/DM-3196

The cleanup to the TPV headers was over-zealous and caused problems for PTF as well as DECam. I've verified that this branch works with PTF and am pulling the DECam example from this ticket, but if someone would like to check at UW that would be OK with me...

OK, I have that tarball downloaded and I still see the problem. I take back the review request...

Let's try again. I have pushed a patch that successfully processes PTF data and works with the tarball that you attached to this ticket.

$ git push

[...]

To git@github.com:lsst/afw

   2cc2aaf..7ccd5fb  tickets/DM-3196 -> tickets/DM-3196

$ git diff --stat origin/master

 src/image/makeWcs.cc | 18 ++++++++----------

 1 file changed, 8 insertions(+), 10 deletions(-)

OK to merge.

Merged to master and pushed

(that took too many edits!)