Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9013

Re-enable MKL/OpenBLAS

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_mosaic
    • Labels:
      None
    • Story Points:
      1
    • Team:
      Data Release Production

      Description

      Use of MKL/OpenBLAS was disabled (in commit 6fe95ec) while adapting meas_mosaic to work with the LSST pipeline. It needs to be re-enabled so we're not limited to the slow matrix inversion using Eigen.

      Reverting that single commit is sufficient to get the threaded matrix inversion:

        PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           
      24920 pprice    32  12 14.7g  13g  45m R 717.0 14.3 506:09.03 python            
      

        Attachments

          Issue Links

            Activity

            Hide
            price Paul Price added a comment -

            Lauren MacArthur, would you please sign off on this?

            price@price-laptop:~/LSST/meas/mosaic (tickets/DM-9013=) $ git sub
            commit cb760dfe14e031e441521656813a91a88b22f264
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Tue Jan 17 14:34:25 2017 -0500
             
                Revert "Force use of Eigen over MKL"
                
                This reverts commit 6fe95ec532dbf6369181d383793ecb8ce824c901.
                
                HSC mosaicking is done using MKL and/or OpenBLAS, and without
                the threading the matrix inversion takes about a Hubble time.
             
             src/mosaicfit.cc | 8 ++++++--
             1 file changed, 6 insertions(+), 2 deletions(-)
            

            Show
            price Paul Price added a comment - Lauren MacArthur , would you please sign off on this? price@price-laptop:~/LSST/meas/mosaic (tickets/DM-9013=) $ git sub commit cb760dfe14e031e441521656813a91a88b22f264 Author: Paul Price <price@astro.princeton.edu> Date: Tue Jan 17 14:34:25 2017 -0500   Revert "Force use of Eigen over MKL" This reverts commit 6fe95ec532dbf6369181d383793ecb8ce824c901. HSC mosaicking is done using MKL and/or OpenBLAS, and without the threading the matrix inversion takes about a Hubble time.   src/mosaicfit.cc | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
            Hide
            lauren Lauren MacArthur added a comment -

            Minor comment on PR. Otherwise, looks good to me.

            Show
            lauren Lauren MacArthur added a comment - Minor comment on PR. Otherwise, looks good to me.
            Hide
            tjenness Tim Jenness added a comment -

            How does this relate to DM-5123? We are deliberately disabling MKL in our conda systems because it crashes with fftw. Is this MKL not the same as the other MKL?

            Show
            tjenness Tim Jenness added a comment - How does this relate to DM-5123 ? We are deliberately disabling MKL in our conda systems because it crashes with fftw. Is this MKL not the same as the other MKL?
            Hide
            price Paul Price added a comment -

            meas_mosaic will use MKL or OpenBLAS to do a matrix inversion if it can find the libraries at run-time. It does not rely on MKL in Conda, but uses whatever MKL or OpenBLAS is available, or none. This therefore shouldn't have any relation to DM-5123.

            Show
            price Paul Price added a comment - meas_mosaic will use MKL or OpenBLAS to do a matrix inversion if it can find the libraries at run-time. It does not rely on MKL in Conda, but uses whatever MKL or OpenBLAS is available, or none. This therefore shouldn't have any relation to DM-5123 .
            Hide
            price Paul Price added a comment -

            On the GitHub PR, Lauren MacArthur pointed out that reverting the commit brings back some code in a style that doesn't fully comply with the LSST style guide. In a private discussion, we agreed that this isn't a concern because the whole codebase has problems, and it's helpful for the commit to exactly revert the former commit.

            Show
            price Paul Price added a comment - On the GitHub PR, Lauren MacArthur pointed out that reverting the commit brings back some code in a style that doesn't fully comply with the LSST style guide. In a private discussion, we agreed that this isn't a concern because the whole codebase has problems, and it's helpful for the commit to exactly revert the former commit.
            Hide
            price Paul Price added a comment -

            Merged to master.

            Show
            price Paul Price added a comment - Merged to master.

              People

              • Assignee:
                price Paul Price
                Reporter:
                price Paul Price
                Reviewers:
                Lauren MacArthur
                Watchers:
                Lauren MacArthur, Paul Price, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel