Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-752

Add the dustmaps package as a dependency of the distributed stack

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      This RFC is asking to:

      1. Add the python package dustmaps to the set of third party stack dependencies
      2. Add a package called dustmaps_cachedata that holds a cached copy of data so that all users do not need fetch a cached version on their own

      Details:
      There are at least two packages that use the dustmaps package: faro and pipe_analysis. In the case of faro we would like to add it to lsst_distrib in the near future (that will be an upcoming RFC). Because the unit tests will not run without dustmaps in the dependency tree, will need to add it before we add faro.

      The one complication is that the dustmaps package requires data to be cached on disk to work. My proposed solution to this, so that not everyone has to cache the data themselves is to make a data package using git-lfs to store cached data that we can use. Here is a stub of a repository that I plan to add to lsst_distrib at the same time. I plan to only add the sfd maps, but if people think we should cache more than that, let me know.

      The dustmaps_cachedata package will set the DUSTMAPS_CONFIG_FNAME to a path within that package containing a config file that points to the data directory in that same package. In order for that to work, I will need to get this this pull request merged, but I have worked with the maintainer before and I don't expect it will be an issue.

        Attachments

          Issue Links

            Activity

            Hide
            krughoff Simon Krughoff added a comment -

            This was already "board recommended" when I made the suggestion of changing how we do this technically. Do I need to put it back into "Flagged" or may I just proceed?

            Show
            krughoff Simon Krughoff added a comment - This was already "board recommended" when I made the suggestion of changing how we do this technically. Do I need to put it back into "Flagged" or may I just proceed?
            Hide
            ktl Kian-Tat Lim added a comment -

            The CCB has accepted the result; I'd say it's up to you to choose the best implementation, so you can just go ahead.

            Show
            ktl Kian-Tat Lim added a comment - The CCB has accepted the result; I'd say it's up to you to choose the best implementation, so you can just go ahead.
            Hide
            tjenness Tim Jenness added a comment -

            Simon Krughoff can you please add an implementation ticket "is triggering"?

            Show
            tjenness Tim Jenness added a comment - Simon Krughoff can you please add an implementation ticket "is triggering"?
            Hide
            krughoff Simon Krughoff added a comment -

            As discussed, I have compressed the SFD maps and put them into the repository proper rather than making it git-lfs.

            Show
            krughoff Simon Krughoff added a comment - As discussed, I have compressed the SFD maps and put them into the repository proper rather than making it git-lfs.
            Hide
            krughoff Simon Krughoff added a comment -

            This is done. The dustmaps_cachedata package is now in our repos.yaml file and the dustmaps module is now a stack dependency. Note that dustmaps_cachedata will still not be distributed with eups distrib, or built in the nightly since no other packages depend on it. That will change when RFC-753 is adopted and faro is added to lsst_apps.

            Show
            krughoff Simon Krughoff added a comment - This is done. The dustmaps_cachedata package is now in our repos.yaml file and the dustmaps module is now a stack dependency. Note that dustmaps_cachedata will still not be distributed with eups distrib , or built in the nightly since no other packages depend on it. That will change when RFC-753 is adopted and faro is added to lsst_apps .

              People

              Assignee:
              krughoff Simon Krughoff
              Reporter:
              krughoff Simon Krughoff
              Watchers:
              Colin Slater, Kian-Tat Lim, Lauren MacArthur, Simon Krughoff, Sophie Reed, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  Jenkins

                  No builds found.