Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10161

Remove maxtasksperchild=1 during pool initialization in meas_mosaic

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_mosaic
    • Labels:
      None
    • Story Points:
      1
    • Epic Link:
    • Sprint:
      DRP S17-5
    • Team:
      Data Release Production

      Description

      Testing for DM-10043 has revealed that having maxtasksperchild=1 in the pool initialization for mosaic.py (the pool being used to read input catalogs in parallel) causes a bug where mosaic.py hangs indefinitely. Curiously, this happens only in with the LSST stack (pybind11 version), not with the HSC stack. It has been demonstrated that removing this constraint removes this hanging behavior.

        Attachments

          Issue Links

            Activity

            tmorton Tim Morton created issue -
            hchiang2 Hsin-Fang Chiang made changes -
            Field Original Value New Value
            Link This issue blocks DM-10129 [ DM-10129 ]
            tmorton Tim Morton made changes -
            Reviewers Jim Bosch [ jbosch ]
            Status To Do [ 10001 ] In Review [ 10004 ]
            tmorton Tim Morton made changes -
            Comment [ This is the one-line fix of the pool keyword argument we have been discussing, enabling mosaic.py to run on the pybind11 stack. ]
            Hide
            swinbank John Swinbank added a comment -

            Before merging, please file another ticket to investigate what's going on here.

            Show
            swinbank John Swinbank added a comment - Before merging, please file another ticket to investigate what's going on here.
            tmorton Tim Morton made changes -
            Link This issue relates to DM-10183 [ DM-10183 ]
            Hide
            jbosch Jim Bosch added a comment -

            I still have no idea what this does, but I'm quite willing to sign off on this on the basis that it's effectively a reversion to old behavior that we will let us move forward, and we'll investigate why it caused problems later.

            Show
            jbosch Jim Bosch added a comment - I still have no idea what this does, but I'm quite willing to sign off on this on the basis that it's effectively a reversion to old behavior that we will let us move forward, and we'll investigate why it caused problems later.
            jbosch Jim Bosch made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            Hide
            tmorton Tim Morton added a comment -

            OK, just merged.

            Show
            tmorton Tim Morton added a comment - OK, just merged.
            tmorton Tim Morton made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            swinbank John Swinbank made changes -
            Epic Link DM-8299 [ 27821 ]
            swinbank John Swinbank made changes -
            Sprint DRP S17-5 [ 364 ]
            Story Points 0.25 1
            Team Data Release Production [ 10301 ]
            swinbank John Swinbank made changes -
            Component/s meas_mosaic [ 10742 ]

              People

              • Assignee:
                tmorton Tim Morton
                Reporter:
                tmorton Tim Morton
                Reviewers:
                Jim Bosch
                Watchers:
                Jim Bosch, John Swinbank, Tim Morton
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel