Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14244

eups.lsst.codes backups not pruning promptly?

    Details

    • Templates:
    • Story Points:
      1
    • Epic Link:
    • Team:
      SQuaRE

      Description

      Daily backups are supposed to be expire out of the s3 bucket at 8 days. This appears to not be happening promptly. This might be because s3 isn't doing this promptly or it might be because it is a versioned bucket. This was noticed as the s3 usage has grown a bit faster than expected.

      $ aws s3 ls s3://eups.lsst.codes-backups/daily/2018/04/
                                 PRE 22/
                                 PRE 23/
                                 PRE 24/
                                 PRE 25/
                                 PRE 26/
                                 PRE 27/
                                 PRE 28/
                                 PRE 29/
                                 PRE 30/
      

      The metadata on the object itself seems to suggest that it will expire in ~9 hours:

       aws s3api head-object --bucket eups.lsst.codes-backups --key daily/2018/04/22/2018-04-22T11:52:06Z/stack/src/tags/w_latest.list
      {
          "AcceptRanges": "bytes",
          "Expiration": "expiry-date=\"Tue, 01 May 2018 00:00:00 GMT\", rule-id=\"daily\"",
          "LastModified": "Sun, 22 Apr 2018 13:55:01 GMT",
          "ContentLength": 6857,
          "ETag": "\"09b6761b4c732b4396521e78bc0dad4c\"",
          "VersionId": "B19u2Pq35xxGQRX06FUfY1SVEFEEKRvh",
          "ContentType": "binary/octet-stream",
          "Metadata": {}
      }
      

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  jhoblitt Joshua Hoblitt
                  Reporter:
                  jhoblitt Joshua Hoblitt
                  Reviewers:
                  Joshua Hoblitt
                  Watchers:
                  Joshua Hoblitt
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  1 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Summary Panel