Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-874

Add pytorch to rubin-env conda defaults

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      AP needs to apply trained machine-learned models in order to provide the required spuriousness scores for DIASources. We have prototype models now ready for deployment in Science Pipelines, but to do so we need to have pytorch added to the default conda environment.

      As discussed in https://jira.lsstcorp.org/browse/RFC-865, pytorch appears to solve successfully with the current rubin-env version.

        Attachments

          Issue Links

          is triggering

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-35938 Add pytorch to rubin-env conda defaults

          • Done
          relates to

          RFC - An issue whose primary purpose is to advertise an intent and allow stakeholders to comment. RFC-865 Switch our conda environment to use mkl by default

          • Withdrawn

          RFC - An issue whose primary purpose is to advertise an intent and allow stakeholders to comment. RFC-909 Addition of TransiNet to the stack

          • Implemented
          mentioned in

          Page Loading...

          Page Loading...

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            ebellm Eric Bellm added a comment -

            Flagging for the DM-CCB.

            Show
            ebellm Eric Bellm added a comment - Flagging for the DM-CCB.
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            I think the biggest question is whether pytorch and mkl are available on ARM Macs and whether mkl will result in any performance degradation on ARM. We do not yet officially support ARM, but I expect that's coming, otherwise people won't be able to use the science pipelines on MacOS at all.

            Show
            Parejkoj John Parejko added a comment - I think the biggest question is whether pytorch and mkl are available on ARM Macs and whether mkl will result in any performance degradation on ARM. We do not yet officially support ARM, but I expect that's coming, otherwise people won't be able to use the science pipelines on MacOS at all.
            Hide
            Permalink
            ebellm Eric Bellm added a comment - - edited

            John Parejko: As Eli Rykoff commented on RFC-685, pytorch is available on ARM macs. mkl does not appear to be, but it is not needed by pytorch on ARM. If (as you said on that RFC) pytorch needs mkl on our USDF machines, it should resolve as a dependency on that platform only without being required on ARM, right?

            In other words, it looks like mamba create -n rumkltest rubin-env=4.0.1 pytorch solves fine both at NCSA and on ARM macs.

            Show
            ebellm Eric Bellm added a comment - - edited John Parejko : As Eli Rykoff commented on RFC-685 , pytorch is available on ARM macs. mkl does not appear to be, but it is not needed by pytorch on ARM. If (as you said on that RFC) pytorch needs mkl on our USDF machines, it should resolve as a dependency on that platform only without being required on ARM, right? In other words, it looks like mamba create -n rumkltest rubin-env=4.0.1 pytorch solves fine both at NCSA and on ARM macs.
            Hide
            Permalink
            ebellm Eric Bellm added a comment -

            Implementation ticket: https://jira.lsstcorp.org/browse/DM-35938

            Show
            ebellm Eric Bellm added a comment - Implementation ticket: https://jira.lsstcorp.org/browse/DM-35938

              People

              Assignee:
              ebellm Eric Bellm
              Reporter:
              ebellm Eric Bellm
              Watchers:
              Eli Rykoff, Eric Bellm, Ian Sullivan, John Parejko, Krzysztof Findeisen, Nima Sedaghat
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  Jenkins

                  No builds found.