Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-22818

Matplotlib 3.1 bug triggered by fgcmcal on macOS

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: fgcmcal
    • Labels:
      None
    • Story Points:
      1
    • Team:
      Architecture

      Description

      I'm testing an updated conda environment. fgcmcal didn't work because it got into an infinite cycle of C++ exceptions.

      fgcmCalibrateTract INFO: Flagging 0 of 472 stars with TOO_FEW_OBS
      libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Couldn't close file
      Caught signal 6, backtrace follows:
      0   libutils.dylib                      0x000000010e657c9f lsst::utils::(anonymous namespace)::signalHandler(int) + 79
      1   libsystem_platform.dylib            0x00007fff6f1b942d (null) + 29
      1   libsystem_platform.dylib            0x00007fff6f1b942d _sigtramp + 29
      2   ???                                 0x075ce2d000000400 0x0 + 530548239103951872
      3   libsystem_c.dylib                   0x00007fff6f08ea1c abort + 120
      4   libc++abi.1.0.dylib                 0x000000010e5e9fa3 abort_message + 195
      5   libc++abi.1.0.dylib                 0x000000010e5b4bcd default_terminate_handler() + 237
      6   libc++abi.1.0.dylib                 0x000000010e5e8ad8 std::__terminate(void (*)()) + 8
      7   libc++abi.1.0.dylib                 0x000000010e5ebbe9 (null) + 121
      7   libc++abi.1.0.dylib                 0x000000010e5ebbe9 __cxa_throw + 121
      8   ft2font.cpython-37m-darwin.so       0x000000010e3e2ced close_file_callback(FT_StreamRec_*) + 573
      9   libfreetype.6.dylib                 0x000000010e3fb846 destroy_face + 582
      10  libfreetype.6.dylib                 0x000000010e3fb57a FT_Done_Face + 170
      11  ft2font.cpython-37m-darwin.so       0x000000010e3da8ce FT2Font::~FT2Font() + 94
      12  ft2font.cpython-37m-darwin.so       0x000000010e3da84f FT2Font::~FT2Font() + 15
      13  ft2font.cpython-37m-darwin.so       0x000000010e3e2218 PyFT2Font_dealloc(PyFT2Font*) + 24
      14  python                              0x0000000103b72d43 lru_list_elem_dealloc + 51
      15  python                              0x0000000103b72e19 lru_cache_cache_clear + 105
      16  python                              0x00000001039757b7 (null) + 135
      16  python                              0x00000001039757b7 _PyMethodDef_RawFastCallDict + 135
      17  python                              0x000000010397506f (null) + 111
      17  python                              0x000000010397506f _PyObject_FastCallDict + 111
      18  python                              0x0000000103b39707 run_at_forkers + 87
      19  python                              0x0000000103b42985 os_fork + 293
      20  python                              0x00000001039766f3 (null) + 131
      20  python                              0x00000001039766f3 _PyMethodDef_RawFastCallKeywords + 131
      21  python                              0x0000000103ab2c82 call_function + 306
      22  python                              0x0000000103aafa36 (null) + 42246
      22  python                              0x0000000103aafa36 _PyEval_EvalFrameDefault + 42246
      23  python                              0x0000000103975ef5 function_code_fastcall + 117
      24  python                              0x0000000103ab2c07 call_function + 183
      25  python                              0x0000000103aaf9a0 (null) + 42096
      25  python                              0x0000000103aaf9a0 _PyEval_EvalFrameDefault + 42096
      26  python                              0x0000000103975ef5 function_code_fastcall + 117
      27  python                              0x00000001039f7591 slot_tp_init + 193
      28  python                              0x0000000103a01691 type_call + 241
      29  python                              0x0000000103976113 (null) + 179
      29  python                              0x0000000103976113 _PyObject_FastCallKeywords + 179
      30  python                              0x0000000103ab2d15 call_function + 453
      31  python                              0x0000000103ab0978 (null) + 46152
      31  python                              0x0000000103ab0978 _PyEval_EvalFrameDefault + 46152
      32  python                              0x0000000103975ef5 function_code_fastcall + 117
      33  python                              0x0000000103ab2c07 call_function + 183
      34  python                              0x0000000103aafa36 (null) + 42246
      34  python                              0x0000000103aafa36 _PyEval_EvalFrameDefault + 42246
      35  python                              0x0000000103975ef5 function_code_fastcall + 117
      36  python                              0x0000000103ab2c07 call_function + 183
      37  python                              0x0000000103aaf9a0 (null) + 42096
      37  python                              0x0000000103aaf9a0 _PyEval_EvalFrameDefault + 42096
      38  python                              0x0000000103975ef5 function_code_fastcall + 117
      39  python                              0x0000000103ab2c07 call_function + 183
      40  python                              0x0000000103aaf9a0 (null) + 42096
      40  python                              0x0000000103aaf9a0 _PyEval_EvalFrameDefault + 42096
      41  python                              0x0000000103aa42ee (null) + 414
      41  python                              0x0000000103aa42ee _PyEval_EvalCodeWithName + 414
      42  python                              0x00000001039753e7 (null) + 231
      42  python                              0x00000001039753e7 _PyFunction_FastCallDict + 231
      43  python                              0x00000001039f7591 slot_tp_init + 193
      44  python                              0x0000000103a01691 type_call + 241
      45  python                              0x0000000103976113 (null) + 179
      45  python                              0x0000000103976113 _PyObject_FastCallKeywords + 179
      46  python                              0x0000000103ab2d15 call_function + 453
      47  python                              0x0000000103ab0a31 (null) + 46337
      47  python                              0x0000000103ab0a31 _PyEval_EvalFrameDefault + 46337
      48  python                              0x0000000103aa42ee (null) + 414
      48  python                              0x0000000103aa42ee _PyEval_EvalCodeWithName + 414
      49  python                              0x0000000103976623 (null) + 195
      49  python                              0x0000000103976623 _PyFunction_FastCallKeywords + 195
      50  python                              0x0000000103ab2c07 call_function + 183
      51  python                              0x0000000103ab0a31 (null) + 46337
      51  python                              0x0000000103ab0a31 _PyEval_EvalFrameDefault + 46337
      52  python                              0x0000000103aa42ee (null) + 414
      52  python                              0x0000000103aa42ee _PyEval_EvalCodeWithName + 414
      53  python                              0x0000000103976623 (null) + 195
      53  python                              0x0000000103976623 _PyFunction_FastCallKeywords + 195
      54  python                              0x0000000103ab2c07 call_function + 183
      55  python                              0x0000000103ab0a31 (null) + 46337
      55  python                              0x0000000103ab0a31 _PyEval_EvalFrameDefault + 46337
      56  python                              0x0000000103975ef5 function_code_fastcall + 117
      57  python                              0x0000000103ab2c07 call_function + 183
      58  python                              0x0000000103aaf9a0 (null) + 42096
      58  python                              0x0000000103aaf9a0 _PyEval_EvalFrameDefault + 42096
      59  python                              0x0000000103975ef5 function_code_fastcall + 117
      60  python                              0x0000000103ab0ba5 (null) + 46709
      60  python                              0x0000000103ab0ba5 _PyEval_EvalFrameDefault + 46709
      61  python                              0x0000000103aa42ee (null) + 414
      61  python                              0x0000000103aa42ee _PyEval_EvalCodeWithName + 414
      62  python                              0x0000000103976623 (null) + 195
      62  python                              0x0000000103976623 _PyFunction_FastCallKeywords + 195
      63  python                              0x0000000103ab2c07 call_function + 183
      64  python                              0x0000000103aaf9a0 (null) + 42096
      64  python                              0x0000000103aaf9a0 _PyEval_EvalFrameDefault + 42096
      65  python                              0x0000000103975ef5 function_code_fastcall + 117
      66  python                              0x00000001039f567d slot_tp_call + 189
      67  python                              0x0000000103976113 (null) + 179
      67  python                              0x0000000103976113 _PyObject_FastCallKeywords + 179
      68  python                              0x0000000103ab2d15 call_function + 453
      69  python                              0x0000000103ab0978 (null) + 46152
      69  python                              0x0000000103ab0978 _PyEval_EvalFrameDefault + 46152
      70  python                              0x0000000103975ef5 function_code_fastcall + 117
      71  python                              0x0000000103ab2c07 call_function + 183
      72  python                              0x0000000103aaf9a0 (null) + 42096
      72  python                              0x0000000103aaf9a0 _PyEval_EvalFrameDefault + 42096
      73  python                              0x0000000103aa42ee (null) + 414
      73  python                              0x0000000103aa42ee _PyEval_EvalCodeWithName + 414
      74  python                              0x0000000103976623 (null) + 195
      74  python                              0x0000000103976623 _PyFunction_FastCallKeywords + 195
      75  python                              0x0000000103ab2c07 call_function + 183
      76  python                              0x0000000103ab0a31 (null) + 46337
      76  python                              0x0000000103ab0a31 _PyEval_EvalFrameDefault + 46337
      77  python                              0x0000000103975ef5 function_code_fastcall + 117
      78  python                              0x0000000103ab2c07 call_function + 183
      79  python                              0x0000000103aaf9a0 (null) + 42096
      79  python                              0x0000000103aaf9a0 _PyEval_EvalFrameDefault + 42096
      80  python                              0x0000000103975ef5 function_code_fastcall + 117
      81  python                              0x0000000103ab2c07 call_function + 183
      82  python                              0x0000000103ab0978 (null) + 46152
      82  python                              0x0000000103ab0978 _PyEval_EvalFrameDefault + 46152
      83  python                              0x0000000103aa42ee (null) + 414
      83  python                              0x0000000103aa42ee _PyEval_EvalCodeWithName + 414
      84  python                              0x00000001039753e7 (null) + 231
      84  python                              0x00000001039753e7 _PyFunction_FastCallDict + 231
      85  python                              0x0000000103979312 method_call + 130
      86  python                              0x0000000103976d92 PyObject_Call + 130
      87  python                              0x0000000103ab0ba5 (null) + 46709
      87  python                              0x0000000103ab0ba5 _PyEval_EvalFrameDefault + 46709
      88  python                              0x0000000103aa42ee (null) + 414
      88  python                              0x0000000103aa42ee _PyEval_EvalCodeWithName + 414
      89  python                              0x00000001039753e7 (null) + 231
      89  python                              0x00000001039753e7 _PyFunction_FastCallDict + 231
      90  python                              0x00000001039f567d slot_tp_call + 189
      91  python                              0x0000000103976113 (null) + 179
      91  python                              0x0000000103976113 _PyObject_FastCallKeywords + 179
      92  python                              0x0000000103ab2d15 call_function + 453
      93  python                              0x0000000103ab0978 (null) + 46152
      93  python                              0x0000000103ab0978 _PyEval_EvalFrameDefault + 46152
      94  python                              0x0000000103aa42ee (null) + 414
      94  python                              0x0000000103aa42ee _PyEval_EvalCodeWithName + 414
      95  python                              0x00000001039753e7 (null) + 231
      95  python                              0x00000001039753e7 _PyFunction_FastCallDict + 231
      96  python                              0x0000000103979312 method_call + 130
      97  python                              0x0000000103976d92 PyObject_Call + 130
      98  python                              0x0000000103ab0ba5 (null) + 46709
      98  python                              0x0000000103ab0ba5 _PyEval_EvalFrameDefault + 46709
      99  python                              0x0000000103aa42ee (null) + 414
      99  python                              0x0000000103aa42ee _PyEval_EvalCodeWithName + 414
      100 python                              0x00000001039753e7 (null) + 231
      100 python                              0x00000001039753e7 _PyFunction_FastCallDict + 231
      101 python                              0x00000001039f567d slot_tp_call + 189
      102 python                              0x0000000103976113 (null) + 179
      102 python                              0x0000000103976113 _PyObject_FastCallKeywords + 179
      103 python                              0x0000000103ab2d15 call_function + 453
      104 python                              0x0000000103ab0978 (null) + 46152
      104 python                              0x0000000103ab0978 _PyEval_EvalFrameDefault + 46152
      105 python                              0x0000000103975ef5 function_code_fastcall + 117
      106 python                              0x0000000103ab2c07 call_function + 183
      107 python                              0x0000000103aaf9a0 (null) + 42096
      107 python                              0x0000000103aaf9a0 _PyEval_EvalFrameDefault + 42096
      108 python                              0x0000000103975ef5 function_code_fastcall + 117
      109 python                              0x0000000103ab2c07 call_function + 183
      110 python                              0x0000000103aaf9a0 (null) + 42096
      110 python                              0x0000000103aaf9a0 _PyEval_EvalFrameDefault + 42096
      111 python                              0x0000000103aa42ee (null) + 414
      111 python                              0x0000000103aa42ee _PyEval_EvalCodeWithName + 414
      112 python                              0x00000001039753e7 (null) + 231
      112 python                              0x00000001039753e7 _PyFunction_FastCallDict + 231
      113 python                              0x00000001039f7591 slot_tp_init + 193
      114 python                              0x0000000103a01691 type_call + 241
      115 python                              0x0000000103976113 (null) + 179
      115 python                              0x0000000103976113 _PyObject_FastCallKeywords + 179
      116 python                              0x0000000103ab2d15 call_function + 453
      117 python                              0x0000000103aafa36 (null) + 42246
      117 python                              0x0000000103aafa36 _PyEval_EvalFrameDefault + 42246
      118 python                              0x0000000103aa42ee (null) + 414
      118 python                              0x0000000103aa42ee _PyEval_EvalCodeWithName + 414
      119 python                              0x0000000103b07b30 PyRun_FileExFlags + 256
      120 python                              0x0000000103b06fa7 PyRun_SimpleFileExFlags + 391
      121 python                              0x0000000103b34c03 pymain_main + 9635
      122 python                              0x0000000103948b4d main + 125
      123 libdyld.dylib                       0x00007fff6efc07fd start + 1
      124 ???                                 0x0000000000000002 0x0 + 2
      

      This was when running tests/test_fgcmcal_hsc.py.

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment - - edited

            Instructions to reproduce:

            $ git clone https://github.com/lsst/lsstsw
            $ cd lsstsw
            $ export LSST_MINICONDA_VERSION=latest
            $ export LSST_SPLENV_REF=master
            $ ./bin/deploy -b
            < wait >
            $ source bin/setup.sh
            $ conda update -n base -c defaults conda
            $ rebuild -r tickets/DM-22818 fgcmcal
            

            Show
            tjenness Tim Jenness added a comment - - edited Instructions to reproduce: $ git clone https://github.com/lsst/lsstsw $ cd lsstsw $ export LSST_MINICONDA_VERSION=latest $ export LSST_SPLENV_REF=master $ ./bin/deploy -b < wait > $ source bin/setup.sh $ conda update -n base -c defaults conda $ rebuild -r tickets/DM-22818 fgcmcal
            Hide
            tjenness Tim Jenness added a comment -

            Eli Rykoff tested fgcmcal on Linux and it worked fine. Given that the back trace involved the font library he suggested that the problem might be related to the plotting in those tests. Running each test separately is fine but running all tests causes the problem. Downgrading my matplotlib to 3.0.3 (the current baseline version) fixed the problem and all tests completed. Looking at the matplotlib GitHub page I find that this problem has already been reported as being a combination of Catalina and multiprocessing: https://github.com/matplotlib/matplotlib/issues/15410

            Show
            tjenness Tim Jenness added a comment - Eli Rykoff tested fgcmcal on Linux and it worked fine. Given that the back trace involved the font library he suggested that the problem might be related to the plotting in those tests. Running each test separately is fine but running all tests causes the problem. Downgrading my matplotlib to 3.0.3 (the current baseline version) fixed the problem and all tests completed. Looking at the matplotlib GitHub page I find that this problem has already been reported as being a combination of Catalina and multiprocessing: https://github.com/matplotlib/matplotlib/issues/15410
            Hide
            tjenness Tim Jenness added a comment -

            Since we have determined that this ticket is not really related to fgcmcal, can you please review the small code cleanups I did make?

            Show
            tjenness Tim Jenness added a comment - Since we have determined that this ticket is not really related to fgcmcal, can you please review the small code cleanups I did make?
            Hide
            erykoff Eli Rykoff added a comment -

            All fine.

            Show
            erykoff Eli Rykoff added a comment - All fine.
            Hide
            tjenness Tim Jenness added a comment -

            Closing this since there is nothing further we can do and we now must wait for matplotlib to fix it.

            Show
            tjenness Tim Jenness added a comment - Closing this since there is nothing further we can do and we now must wait for matplotlib to fix it.

              People

              • Assignee:
                tjenness Tim Jenness
                Reporter:
                tjenness Tim Jenness
                Reviewers:
                Eli Rykoff
                Watchers:
                Eli Rykoff, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel