Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4944

butler should transparently allow files to be compressed or not

    Details

    • Type: Story
    • Status: Invalid
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: butler
    • Labels:
      None

      Description

      see the conversation on c.l.o. at https://community.lsst.org/t/how-does-the-butler-support-compression/502.

      The summary is, when the mapper returns e.g. a non compressed file name e.g. foo.fits, that file may be compressed and the filename may reflect this e.g. in reality it might be named foo.fits.gz. On a posix system some component of the butler framework should discover this and transform the filename to the correct filename and pass that to the deserializer.

      TBD if the list of allowed extensions is hard coded someplace (in a mapper subclass?) or specified another way, perhaps by the policy (could be for dataset type or globally).

        Attachments

          Issue Links

            Activity

            Hide
            npease Nate Pease added a comment -

            Robert Lupton is this needed "right now", "asap", or "sometime soonish"?

            Show
            npease Nate Pease added a comment - Robert Lupton is this needed "right now", "asap", or "sometime soonish"?
            Hide
            rhl Robert Lupton added a comment -

            I can fake things by monkey-patching butler.get, so I can't claim that it's super-urgent.

            And even if it were super-urgent for me, it's still Jacek's call.

            Show
            rhl Robert Lupton added a comment - I can fake things by monkey-patching butler.get, so I can't claim that it's super-urgent. And even if it were super-urgent for me, it's still Jacek's call.
            Hide
            npease Nate Pease added a comment -

            this was done as DM-8254

            Show
            npease Nate Pease added a comment - this was done as DM-8254

              People

              • Assignee:
                Unassigned
                Reporter:
                npease Nate Pease
                Watchers:
                Nate Pease, Robert Lupton
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel