Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16061

Fix runtime error with LTD Keeper's delete_objects function

    XMLWordPrintable

    Details

      Description

      I'm observing a new runtime error in LTD Keeper. This happened when trying to rebuild the master edition of ldm-538 to use the latest existing build from the master branch.

      {
       insertId:  "1gfyt0rg1lpguyc"  
       labels: {…}  
       logName:  "projects/plasma-geode-127520/logs/keeper-worker"  
       receiveTimestamp:  "2018-10-09T16:11:09.569179750Z"  
       resource: {…}  
       severity:  "ERROR"  
       textPayload:  "Traceback (most recent call last):
        File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 382, in trace_task
          R = retval = fun(*args, **kwargs)
        File "/usr/local/lib/python3.6/site-packages/keeper/celery.py", line 35, in __call__
          return TaskBase.__call__(self, *args, **kwargs)
        File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 641, in __protected_call__
          return self.run(*args, **kwargs)
        File "/usr/local/lib/python3.6/site-packages/keeper/tasks/editionrebuild.py", line 60, in rebuild_edition
          cache_control='no-cache')
        File "/usr/local/lib/python3.6/site-packages/keeper/s3.py", line 147, in copy_directory
          aws_access_key_id, aws_secret_access_key)
        File "/usr/local/lib/python3.6/site-packages/keeper/s3.py", line 54, in delete_directory
          keys['Objects'].append({'Key': item['Key']})
      TypeError: 'NoneType' object is not subscriptable
      "  
       timestamp:  "2018-10-09T16:11:06Z"  
      }
      

      This is highly related to the new pagination work done in DM-15518.

      For reference, this is the operation that triggered the error:

      http --auth $TOKEN: patch https://keeper.lsst.codes/editions/1903 build_url="https://keeper.lsst.codes/builds/6129"
      

        Attachments

          Issue Links

            Activity

            jsick Jonathan Sick created issue -
            jsick Jonathan Sick made changes -
            Field Original Value New Value
            Epic Link DM-15819 [ 208467 ]
            jsick Jonathan Sick made changes -
            Link This issue relates to DM-15518 [ DM-15518 ]
            jsick Jonathan Sick made changes -
            Risk Score 0
            jsick Jonathan Sick made changes -
            Description I'm observing a new runtime error in LTD Keeper. This happened when trying to rebuild the {{master}} edition of {{ldm-538}} to use the latest existing build from the master branch.

            {code}
            {
             insertId: "1gfyt0rg1lpguyc"
             labels: {…}
             logName: "projects/plasma-geode-127520/logs/keeper-worker"
             receiveTimestamp: "2018-10-09T16:11:09.569179750Z"
             resource: {…}
             severity: "ERROR"
             textPayload: "Traceback (most recent call last):
              File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 382, in trace_task
                R = retval = fun(*args, **kwargs)
              File "/usr/local/lib/python3.6/site-packages/keeper/celery.py", line 35, in __call__
                return TaskBase.__call__(self, *args, **kwargs)
              File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 641, in __protected_call__
                return self.run(*args, **kwargs)
              File "/usr/local/lib/python3.6/site-packages/keeper/tasks/editionrebuild.py", line 60, in rebuild_edition
                cache_control='no-cache')
              File "/usr/local/lib/python3.6/site-packages/keeper/s3.py", line 147, in copy_directory
                aws_access_key_id, aws_secret_access_key)
              File "/usr/local/lib/python3.6/site-packages/keeper/s3.py", line 54, in delete_directory
                keys['Objects'].append({'Key': item['Key']})
            TypeError: 'NoneType' object is not subscriptable
            "
             timestamp: "2018-10-09T16:11:06Z"
            }
            {code}

            This is highly related to the new pagination work done in DM-15518.
            I'm observing a new runtime error in LTD Keeper. This happened when trying to rebuild the {{master}} edition of {{ldm-538}} to use the latest existing build from the master branch.

            {code}
            {
             insertId: "1gfyt0rg1lpguyc"
             labels: {…}
             logName: "projects/plasma-geode-127520/logs/keeper-worker"
             receiveTimestamp: "2018-10-09T16:11:09.569179750Z"
             resource: {…}
             severity: "ERROR"
             textPayload: "Traceback (most recent call last):
              File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 382, in trace_task
                R = retval = fun(*args, **kwargs)
              File "/usr/local/lib/python3.6/site-packages/keeper/celery.py", line 35, in __call__
                return TaskBase.__call__(self, *args, **kwargs)
              File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 641, in __protected_call__
                return self.run(*args, **kwargs)
              File "/usr/local/lib/python3.6/site-packages/keeper/tasks/editionrebuild.py", line 60, in rebuild_edition
                cache_control='no-cache')
              File "/usr/local/lib/python3.6/site-packages/keeper/s3.py", line 147, in copy_directory
                aws_access_key_id, aws_secret_access_key)
              File "/usr/local/lib/python3.6/site-packages/keeper/s3.py", line 54, in delete_directory
                keys['Objects'].append({'Key': item['Key']})
            TypeError: 'NoneType' object is not subscriptable
            "
             timestamp: "2018-10-09T16:11:06Z"
            }
            {code}

            This is highly related to the new pagination work done in DM-15518.

            For reference, this is the operation that triggered the error:

            {code}
            http --auth $TOKEN: patch https://keeper.lsst.codes/editions/1903 build_url="https://keeper.lsst.codes/builds/6129"
            {code}
            jsick Jonathan Sick made changes -
            Story Points 0.7
            jsick Jonathan Sick made changes -
            Resolution Done [ 10000 ]
            Status To Do [ 10001 ] Done [ 10002 ]

              People

              Assignee:
              jsick Jonathan Sick
              Reporter:
              jsick Jonathan Sick
              Watchers:
              Jonathan Sick
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: