Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5123

develop strategy for handling environments with Intel's MKL library

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: afw
    • Labels:
      None
    • Team:
      Architecture

      Description

      Per discussion with Tim Jenness and a bit of symbol table inspection, the root cause of the issue observed in DM-5105 is likely due to the Intel MKL library duplicating symbols which are present in fftw, blas, etc.. Eg.,

      https://software.intel.com/en-us/node/522277

      After discussion with CA sales, it has been clarified that the new mkl conda packages are under different terms than the previous MKL Optimized "add-ons".

      http://docs.continuum.io/anaconda/eula

      The new license is permissive and allows redistribution. It is reasonable to assume that MKL will be present on more LSST enduser's systems in the future and we need a strategy for managing linking to prevent symbol conflicts. A few possibilities are:

      • remove fftw, openblas, etc. from the stack and link solely with MKL (unknown if it is a 100% replacement and this would tie us to the conda package with the permissive eula)
      • officially declare MKL as unsupported
      • create library stubs (ie., symlinks) so that -lfftw/etc. will continue to work and re-plumb how EUPS handles optional dependencies so we can switch between openblas/fftw and MKL
      • modify our scons probing logic to be able to select between the various library options and re-plumb EUPS optional dependency handling

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            Can either of Mario Juric or John Swinbank confirm that this ticket is complete in that we have agreed that the strategy is to fix meas_base in DM-8146 ?

            Show
            tjenness Tim Jenness added a comment - Can either of Mario Juric or John Swinbank confirm that this ticket is complete in that we have agreed that the strategy is to fix meas_base in DM-8146 ?
            Hide
            swinbank John Swinbank added a comment -

            I can confirm that the work on DM-8146 is in the DRP backlog and will be scheduled.

            I don't know anything much about MKL beyond what I've learned from scanning this ticket, so I can't confirm that DM-8146 is a sufficient "strategy for handling environments with Intel's MKL library".

            Show
            swinbank John Swinbank added a comment - I can confirm that the work on DM-8146 is in the DRP backlog and will be scheduled. I don't know anything much about MKL beyond what I've learned from scanning this ticket, so I can't confirm that DM-8146 is a sufficient "strategy for handling environments with Intel's MKL library".
            Hide
            tjenness Tim Jenness added a comment -

            Mario Juric will you be able to look at this?

            Show
            tjenness Tim Jenness added a comment - Mario Juric will you be able to look at this?
            Hide
            mjuric Mario Juric added a comment -

            Hi Tim Jenness, sorry, lost these in JIRA noise.

            Yes, I do agree that DM-8146 is the right (righteous) solution (as opposed to a workaround).

            Show
            mjuric Mario Juric added a comment - Hi Tim Jenness , sorry, lost these in JIRA noise. Yes, I do agree that DM-8146 is the right (righteous) solution (as opposed to a workaround).
            Hide
            tjenness Tim Jenness added a comment -

            Mario Juric thank you. I'll close this now.

            Show
            tjenness Tim Jenness added a comment - Mario Juric thank you. I'll close this now.

              People

              • Assignee:
                tjenness Tim Jenness
                Reporter:
                jhoblitt Joshua Hoblitt
                Reviewers:
                Mario Juric
                Watchers:
                Chris Walter, Dominique Boutigny, Frossie Economou, Jim Bosch, John Parejko, John Swinbank, Joshua Hoblitt, Kian-Tat Lim, Mario Juric, Paul Price, Robert Lupton, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel