Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10161

Remove maxtasksperchild=1 during pool initialization in meas_mosaic

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_mosaic
    • Labels:
      None
    • Story Points:
      1
    • Epic Link:
    • Sprint:
      DRP S17-5
    • Team:
      Data Release Production

      Description

      Testing for DM-10043 has revealed that having maxtasksperchild=1 in the pool initialization for mosaic.py (the pool being used to read input catalogs in parallel) causes a bug where mosaic.py hangs indefinitely. Curiously, this happens only in with the LSST stack (pybind11 version), not with the HSC stack. It has been demonstrated that removing this constraint removes this hanging behavior.

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-10129 Process HSC RC dataset using Stack version chosen for the full S17B reprocessing

          • Done
          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-10183 Investigate why maxtasksperchild=1 causes mosaic.py to hang on pybind11 stack

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            swinbank John Swinbank added a comment -

            Before merging, please file another ticket to investigate what's going on here.

            Show
            swinbank John Swinbank added a comment - Before merging, please file another ticket to investigate what's going on here.
            Hide
            Permalink
            jbosch Jim Bosch added a comment -

            I still have no idea what this does, but I'm quite willing to sign off on this on the basis that it's effectively a reversion to old behavior that we will let us move forward, and we'll investigate why it caused problems later.

            Show
            jbosch Jim Bosch added a comment - I still have no idea what this does, but I'm quite willing to sign off on this on the basis that it's effectively a reversion to old behavior that we will let us move forward, and we'll investigate why it caused problems later.
            Hide
            Permalink
            tmorton Tim Morton [X] (Inactive) added a comment -

            OK, just merged.

            Show
            tmorton Tim Morton [X] (Inactive) added a comment - OK, just merged.

              People

              Assignee:
              tmorton Tim Morton [X] (Inactive)
              Reporter:
              tmorton Tim Morton [X] (Inactive)
              Reviewers:
              Jim Bosch
              Watchers:
              Jim Bosch, John Swinbank, Tim Morton [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.