# Butler should support flags for writing source tables

XMLWordPrintable

#### Details

• Type: Story
• Status: To Do
• Resolution: Unresolved
• Fix Version/s: None
• Component/s:
• Labels:
• Team:
Architecture

#### Description

Simon Krughoff discovered an interesting bug: several tasks that save source catalogs using butler.put want to specify a value for the flags argument of the FITS writer. They do this by passing flags=... to the butler.put.... However, the butler assumes that the flag argument is another data ID key:value pair. (Normally this extra item is ignored, which is why we have only now noticed it, but in some cases it can cause an exception).

We might be able to manage without passing the flag at all, but I worry that is not a good solution. I'd be much happier having some way to pass the flags argument through to the persistence code.

#### Activity

Hide
Simon Krughoff added a comment -

Did this get resolved in any way? I just ran into this again with people reducing DECam data.

Show
Simon Krughoff added a comment - Did this get resolved in any way? I just ran into this again with people reducing DECam data.
Hide
John Parejko added a comment -

I just ran into this with CFHT data, and there was a discussion about it on slack. This is the sort of command that fails:

 data = butler.get("src", dataId=dataId, flags=lsst.afw.table.SOURCE_IO_NO_FOOTPRINTS) 

Can we get this taken care of, please?

Show
John Parejko added a comment - I just ran into this with CFHT data, and there was a discussion about it on slack. This is the sort of command that fails: data = butler.get("src", dataId=dataId, flags=lsst.afw.table.SOURCE_IO_NO_FOOTPRINTS) Can we get this taken care of, please?
Hide
Michael Wood-Vasey [X] (Inactive) added a comment -

Kian-Tat LimWill
A) This ticket be resolved by Gen3?
B) We be able to load a Source Catalog without footprints in Gen3?

Show
Michael Wood-Vasey [X] (Inactive) added a comment - Kian-Tat Lim Will A) This ticket be resolved by Gen3? B) We be able to load a Source Catalog without footprints in Gen3?
Hide
Tim Jenness added a comment -

Gen3 does not allow butler.put to have write flags given by the user. The user has to strip the footprints out before writing.

Flags on read to control footprints are covered by DM-26761.

Show
Tim Jenness added a comment - Gen3 does not allow butler.put to have write flags given by the user. The user has to strip the footprints out before writing. Flags on read to control footprints are covered by DM-26761 .
Hide
Tim Jenness added a comment -

This came up again today on community (https://community.lsst.org/t/do-write-heavy-footprints-in-sources-appears-not-to-have-any-effect/7024). The outcome seems to be that:

• Deciding whether or not to include the footprints on write should be a configuration parameter for the task and not for the butler to decide.
• Controlling whether to read the heavy footprints is still needed.
Show
Tim Jenness added a comment - This came up again today on community ( https://community.lsst.org/t/do-write-heavy-footprints-in-sources-appears-not-to-have-any-effect/7024 ). The outcome seems to be that: Deciding whether or not to include the footprints on write should be a configuration parameter for the task and not for the butler to decide. Controlling whether to read the heavy footprints is still needed.

#### People

Assignee:
Kian-Tat Lim
Reporter:
Russell Owen
Watchers:
Colin Slater, Jim Bosch, John Parejko, Kian-Tat Lim, Michael Wood-Vasey [X] (Inactive), Nate Pease [X] (Inactive), Paul Price, Russell Owen, Simon Krughoff, Tim Jenness