Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12029

recompress jointcal's testdata zeroed images with fpack

    Details

    • Templates:
    • Story Points:
      2
    • Sprint:
      Alert Production F17 - 10
    • Team:
      Alert Production

      Description

      Now that the stack supports tile compression of FITS files, and a number of bugs related to handling of FITS image metadata were fixed (DM-11332), I can recompress the images (really just holders of the metadata) I zeroed out in testdata_jointcal so that they aren't just gzipped files. Once done I can use the new butler calls (DM-9060, DM-9153) to directly access that metadata. This should drastically speed up file ingestion for jointcal.

      The compression step is probably best done via a short bash script that runs gunzip and then fpack, with some carefully chosen parameters (the files are zeroed, so it probably doesn't matter much which compression scheme is selected).

      If this works, we can close DM-6911 as well, as that probably got fixed "for free" as part of DM-11332.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                Parejkoj John Parejko
                Reviewers:
                Paul Price
                Watchers:
                John Parejko, John Swinbank, Paul Price
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel