Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15072

Kombu error with LTD Keeper 1.11.0

    XMLWordPrintable

    Details

      Description

      There may be an incompatibility in an implicit dependency found through Celery that is preventing the LTD Keeper workers from booting up:

      Traceback (most recent call last): File "/usr/local/bin/celery", line 11, in <module> sys.exit(main()) File "/usr/local/lib/python3.5/site-packages/celery/__main__.py", line 14, in main _main() File "/usr/local/lib/python3.5/site-packages/celery/bin/celery.py", line 326, in main cmd.execute_from_commandline(argv) File "/usr/local/lib/python3.5/site-packages/celery/bin/celery.py", line 488, in execute_from_commandline super(CeleryCommand, self).execute_from_commandline(argv))) File "/usr/local/lib/python3.5/site-packages/celery/bin/base.py", line 281, in execute_from_commandline return self.handle_argv(self.prog_name, argv[1:]) File "/usr/local/lib/python3.5/site-packages/celery/bin/celery.py", line 480, in handle_argv return self.execute(command, argv) File "/usr/local/lib/python3.5/site-packages/celery/bin/celery.py", line 412, in execute ).run_from_argv(self.prog_name, argv[1:], command=argv[0]) File "/usr/local/lib/python3.5/site-packages/celery/bin/worker.py", line 221, in run_from_argv return self(*args, **options) File "/usr/local/lib/python3.5/site-packages/celery/bin/base.py", line 244, in __call__ ret = self.run(*args, **kwargs) File "/usr/local/lib/python3.5/site-packages/celery/bin/worker.py", line 255, in run **kwargs) File "/usr/local/lib/python3.5/site-packages/celery/worker/worker.py", line 99, in __init__ self.setup_instance(**self.prepare_args(**kwargs)) File "/usr/local/lib/python3.5/site-packages/celery/worker/worker.py", line 122, in setup_instance self.should_use_eventloop() if use_eventloop is None File "/usr/local/lib/python3.5/site-packages/celery/worker/worker.py", line 241, in should_use_eventloop self._conninfo.transport.implements.async and File "/usr/local/lib/python3.5/site-packages/kombu/transport/base.py", line 125, in __getattr__ raise AttributeError(key) AttributeError: async
      

      https://console.cloud.google.com/logs/viewer?project=plasma-geode-127520&authuser=1&_ga=2.223555801.-353975650.1530195679&pli=1&minLogLevel=0&expandAll=false&timestamp=2018-07-09T22%3A49%3A09.738000000Z&customFacets&limitCustomFacetWidth=true&dateRangeStart=2018-07-09T21%3A49%3A09.990Z&dateRangeEnd=2018-07-09T22%3A49%3A09.990Z&interval=PT1H&resource=container%2Fcluster_name%2Flsst-docs&scrollTimestamp=2018-07-09T22%3A36%3A18.000000000Z&advancedFilter=resource.type%3D%22container%22%0Aresource.labels.pod_id%3D%22keeper-worker-deployment-5457bf656-zrpmf%22%0Aresource.labels.zone%3D%22us-central1-b%22%0Aresource.labels.project_id%3D%22plasma-geode-127520%22%0Aresource.labels.cluster_name%3D%22lsst-docs%22%0Aresource.labels.container_name%3D%22keeper-worker%22%0Aresource.labels.namespace_id%3D%22ltd-prod%22%0Aresource.labels.instance_id%3D%2244590524507142367%22%0Atimestamp%3D%222018-07-09T22%3A36%3A18.000000000Z%22%0AinsertId%3D%22towqstg1iy3niu%22

      Appears with LTD Keeper 1.11.0.

      The tickets-DM-14122 Docker image deploys ok. But the 1.9.0 image does not.

      1.9.0 was built retroactively much later than the associated ticket was merged (and tickets-DM-14122 was built).

      This means I think there was a change in kombu that we're picking up as a floating implicit dependency of the pinned celery dep.

        Attachments

          Issue Links

            Activity

            jsick Jonathan Sick created issue -
            jsick Jonathan Sick made changes -
            Field Original Value New Value
            Epic Link DM-12790 [ 36408 ]
            jsick Jonathan Sick made changes -
            Link This issue is triggered by DM-15016 [ DM-15016 ]
            jsick Jonathan Sick made changes -
            Risk Score 0
            jsick Jonathan Sick made changes -
            Remote Link This issue links to "GitHub issue for Kombu error (Web Link)" [ 17391 ]
            jsick Jonathan Sick made changes -
            Story Points 0.5
            jsick Jonathan Sick made changes -
            Resolution Done [ 10000 ]
            Status To Do [ 10001 ] Done [ 10002 ]

              People

              Assignee:
              jsick Jonathan Sick
              Reporter:
              jsick Jonathan Sick
              Watchers:
              Jonathan Sick
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.