Details
-
Type:
RFC
-
Status: Adopted
-
Resolution: Done
-
Component/s: DM
-
Labels:None
Description
Camera geometry used to be defined using PAF (policy) files, which are now deprecated. As part of the transition to the refactored camera geometry scheme, scripts were introduced to convert from the PAF files to the new camera geometry configuration scheme which uses FITS files and a python file to describe the camera. These scripts ("genCameraRepository.py" in obs_cfht, "makeLsstCameraRepository.py" in obs_lsstSim, "makeDecamCameraRepository.py" in obs_decam) are still part of the obs_* packages, and some people rely on them for making changes to the camera description. On the other hand, the generated FITS files and python file are also first-class members of the obs_* packages. This means that we have two sources of the same information, which is dangerous. What should be viewed as the primary source? The options appear to be:
1. The generated FITS files and python files are primary, and the conversion scripts were temporary, intended for the transition only. This is the approach we have taken in obs_subaru, where we have deleted the conversion script as a relic of the past and track only the FITS tables because that's what the CameraMapper uses; the disadvantage is that changes to the camera geometry require editing the binary FITS tables. If this option is accepted, we should delete the conversion scripts in all obs_* packages.
2. The conversion scripts are primary, and the generated FITS files and python files are just a cache for the CameraMapper to read. This provides a readable and easily editable source, but requires additional steps to generate what the CameraMapper will use. If this option is accepted, we should delete the generated FITS files and python files from the obs_* packages and generate them at build time. We should also update the conversion scripts to use a different source than the deprecated PAF files (e.g., YAML).
3. Some other source is primary. Maybe the FITS files weren't such a great idea because they're hard to edit, and we should specify our camera geometry purely in python. This is the approach taken in obs_monocam. However, Monocam has only one CCD, and maybe this wouldn't be feasible for cameras with a hundred CCDs.
It may be that this choice could be different for different obs_* packages depending on the preferences of the maintainer, but a choice needs to be made for each.
Attachments
Issue Links
Activity
I don't think I made an actual proposal for people to disagree with. I suggest adopting option 1, as I think that state is closest to what was envisioned in the cameraGeom refactoring (at least, my interpretation of what was envisioned) and it's the easiest to implement. I don't mind the other options, but they're a lot more work — they need to invent new configuration schemes, which is exactly one of the reasons we did the cameraGeom refactoring in the first place, and that invented the FITS files so I suggest we use those.
I would prefer that the different obs_* packages all adopt the same solution, as it makes it easier to move between packages.
Paul Price
added a comment - I don't think I made an actual proposal for people to disagree with. I suggest adopting option 1, as I think that state is closest to what was envisioned in the cameraGeom refactoring (at least, my interpretation of what was envisioned) and it's the easiest to implement. I don't mind the other options, but they're a lot more work — they need to invent new configuration schemes, which is exactly one of the reasons we did the cameraGeom refactoring in the first place, and that invented the FITS files so I suggest we use those.
I would prefer that the different obs_* packages all adopt the same solution, as it makes it easier to move between packages. I don't have terribly strong opinions here, but I do agree that this needs to be fixed somehow.
I also think that camera descriptions will inevitably contain both machine-generated binary blobs and human-generated structured text. To me that suggests that we should have a YAML like format for human-generated, human-generated comment and structure that can delegate individual components to binary blobs stored some other way (such as FITS or HDF5).
I'll weigh in here. I really always meant the in memory object to be the first class product and that the on disk representation was an implementation detail, but I understand the pain with updating the fits files. They were meant to be a convenience for getting a camera to the butler. I didn't envision we'd need to update the camera model very often and when we did that it would be in memory.
I think, in reality, we need to allow multiple of the options. The obs_lsstSim mechanism (conversion script), for example, is essentially dictated by how the image simulation software is configured. For cameras like monocam, having a python implementation makes a ton of sense.
Given that I don't think we can choose a single option for all obs_* packages, we can eliminate some confusion by suggesting that any given obs package has only one of the options. I think we should also require that paf files disappear ASAP.
Just my thoughts.
I've created DM-6349 to replace the PAF files. Simon Krughoff, could you please schedule this?
I've created DM-6350 to delete the generated files and have them created at build time. I'm happy to handle that.
Paul Price
added a comment - I've created DM-6349 to replace the PAF files. Simon Krughoff , could you please schedule this?
I've created DM-6350 to delete the generated files and have them created at build time. I'm happy to handle that. Paul Price I'll schedule it.
We agreed to remove the multiple options; this will be done in a different way for each obs_* package. Work has been scheduled.
Paul Price
added a comment - We agreed to remove the multiple options; this will be done in a different way for each obs_* package. Work has been scheduled. 
I agree that the current situation is untenable. I think any of these solutions is acceptable and I suspect different projects may prefer different solutions.
I wish we could use astropy tables instead of afw tables to hold the information (since the tools to manage those are a bit easier to use), but presently the information needs to be stored in a C++ object in the Detector, and I don't see that being changed soon.