Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-34753

Add parameter support for SourceFitsFlags in Catalog FITS formatters

    XMLWordPrintable

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: obs_base
    • Labels:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      The afw.table readFits and writeFits methods can take a SourceFitsFlag flag enumeration to control whether footprints and heavy footprints are read and written, but the formatters don't provide a pass to pass them through. This is particularly problematic for the read case, where not reading footprints when they are not needed is a potentially big optimization.

      It's arguable that these would be better framed as components of some kind rather than parameters (Tim Jenness, any thoughts?), but parameters seems a lot easier, especially when backwards compatibility is in play.

        Attachments

          Activity

          Hide
          tjenness Tim Jenness added a comment -

          There is no problem with adding a read parameter to the Storage Class to instruct the formatter to skip reading the footprints. This is discussed in DM-26761 (which is now a duplicate I think).

          The only way at the moment butler can be told by the pipeline not to write the heavy footprints is for the pipeline to strip the heavy footprints out before putting the dataset. Formatters can't take parameters for writing because all previous requests have been people asking to specify a parameter to control the file format or things like compression options and those should never be something that a butler.put should have to care about because the butler user is not meant to know the format of the file that is being written. Those types of changes are controlled by datastore configuration of Formatters and writeRecipes.

          We could consider a put parameter but it would have to be something that could be implemented by acting on the Python object itself (and so always be supported for the StorageClass regardless of file formatter) and not solely a special feature of a Formatter (get parameters have the same property – the storage class delegate has to be able to handle them if the formatter could not)

          Show
          tjenness Tim Jenness added a comment - There is no problem with adding a read parameter to the Storage Class to instruct the formatter to skip reading the footprints. This is discussed in DM-26761 (which is now a duplicate I think). The only way at the moment butler can be told by the pipeline not to write the heavy footprints is for the pipeline to strip the heavy footprints out before putting the dataset. Formatters can't take parameters for writing because all previous requests have been people asking to specify a parameter to control the file format or things like compression options and those should never be something that a butler.put should have to care about because the butler user is not meant to know the format of the file that is being written. Those types of changes are controlled by datastore configuration of Formatters and writeRecipes. We could consider a put parameter but it would have to be something that could be implemented by acting on the Python object itself (and so always be supported for the StorageClass regardless of file formatter) and not solely a special feature of a Formatter (get parameters have the same property – the storage class delegate has to be able to handle them if the formatter could not)

            People

            Assignee:
            jbosch Jim Bosch
            Reporter:
            jbosch Jim Bosch
            Watchers:
            Jim Bosch, Tim Jenness
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:

                Jenkins Builds

                No builds found.