Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8256

Create lsst_obs package & add to lsst_distrib

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: lsst_distrib
    • Labels:
      None

      Description

      Following on from RFC-251 we need to create a new package called lsst_obs and add all the obs packages to it. This package should then be added to lsst_distrib. The lsst_obs package should be documented to allow broken packages to be removed if they are holding up a release if they are not officially supported by DM.

        Attachments

          Issue Links

            Activity

            Hide
            swinbank John Swinbank added a comment -

            We may wish to consider what "all" the obs packages actually means. I assume at least cfht, decam and subaru, but not e.g. monocam.

            Show
            swinbank John Swinbank added a comment - We may wish to consider what "all" the obs packages actually means. I assume at least cfht, decam and subaru, but not e.g. monocam.
            Hide
            tjenness Tim Jenness added a comment - - edited

            I think Kian-Tat Lim was proposing we go maximal and see what breaks; with the understanding that if something in lsst_obs breaks we are allowed to remove the breaking package temporarily. obs_monocam is in lsst_py3 at the moment.

            Show
            tjenness Tim Jenness added a comment - - edited I think Kian-Tat Lim was proposing we go maximal and see what breaks; with the understanding that if something in lsst_obs breaks we are allowed to remove the breaking package temporarily. obs_monocam is in lsst_py3 at the moment.
            Hide
            ktl Kian-Tat Lim added a comment -

            Yes, distributed "as-is" with no guarantee of merchantability or fitness for a particular purpose other than we managed to build it, and it passed its own tests on our platforms. Borrow some text from a shrink-wrap license for the README

            Show
            ktl Kian-Tat Lim added a comment - Yes, distributed "as-is" with no guarantee of merchantability or fitness for a particular purpose other than we managed to build it, and it passed its own tests on our platforms. Borrow some text from a shrink-wrap license for the README
            Hide
            Parejkoj John Parejko added a comment -

            obs_monocam cannot be broken by definition: it has no tests.

            Does this then include all of the testdata packages?

            Show
            Parejkoj John Parejko added a comment - obs_monocam cannot be broken by definition: it has no tests. Does this then include all of the testdata packages?
            Hide
            tjenness Tim Jenness added a comment -

            The testdata packages will have to be excluded from the publish (they are optional) as we don't want eups distrib users downloading massive amounts of data (just like afwdata is excluded).

            Show
            tjenness Tim Jenness added a comment - The testdata packages will have to be excluded from the publish (they are optional) as we don't want eups distrib users downloading massive amounts of data (just like afwdata is excluded).
            Hide
            Parejkoj John Parejko added a comment -

            Ah! Ok, that makes sense. That method only works for eups distrib installs, not lsstsw, right?

            Show
            Parejkoj John Parejko added a comment - Ah! Ok, that makes sense. That method only works for eups distrib installs, not lsstsw, right?
            Hide
            tjenness Tim Jenness added a comment -

            lsstsw picks up optionals. I think that eups distrib server has an exclusion list containing afwdata but I'm not entirely sure. Or maybe that exclusion list is on the lsst-dev build system that does the rebuilds and publish.

            Show
            tjenness Tim Jenness added a comment - lsstsw picks up optionals. I think that eups distrib server has an exclusion list containing afwdata but I'm not entirely sure. Or maybe that exclusion list is on the lsst-dev build system that does the rebuilds and publish.
            Hide
            ktl Kian-Tat Lim added a comment -

            Or maybe that exclusion list is on the lsst-dev build system that does the rebuilds and publish.

            I believe this is lsst-dev.ncsa.illinois.edu:~lsstsw/etc/manifest.remap.

            Show
            ktl Kian-Tat Lim added a comment - Or maybe that exclusion list is on the lsst-dev build system that does the rebuilds and publish. I believe this is lsst-dev.ncsa.illinois.edu:~lsstsw/etc/manifest.remap .
            Hide
            swinbank John Swinbank added a comment -

            Who is on the hook for this work? It's blocking the 13.0 release so presumably we want to get it done ASAP.

            Show
            swinbank John Swinbank added a comment - Who is on the hook for this work? It's blocking the 13.0 release so presumably we want to get it done ASAP.
            Hide
            tjenness Tim Jenness added a comment -

            John Swinbank I think the situation is that Frossie Economou will try to do it as part of the release packaging but she would be very happy if someone else volunteers the hour to do it.

            Show
            tjenness Tim Jenness added a comment - John Swinbank I think the situation is that Frossie Economou will try to do it as part of the release packaging but she would be very happy if someone else volunteers the hour to do it.
            Hide
            swinbank John Swinbank added a comment -

            Ok, then I'll set the team to SQuaRE.

            Show
            swinbank John Swinbank added a comment - Ok, then I'll set the team to SQuaRE.
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            There aren't repos.yaml entries for these obs_* packages so they are being excluded on the basis they have never been CI'd. obs_sst, obs_wiyn, obs_noao_mosaic

            Show
            jhoblitt Joshua Hoblitt added a comment - There aren't repos.yaml entries for these obs_* packages so they are being excluded on the basis they have never been CI'd. obs_sst, obs_wiyn, obs_noao_mosaic
            Hide
            jhoblitt Joshua Hoblitt added a comment -
            Show
            jhoblitt Joshua Hoblitt added a comment - CI build of lsst_obs : https://ci.lsst.codes/job/stack-os-matrix/19357/
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            Note that new package does not show on issue as a branch: https://github.com/lsst/lsst_obs

            Show
            jhoblitt Joshua Hoblitt added a comment - Note that new package does not show on issue as a branch: https://github.com/lsst/lsst_obs
            Hide
            tjenness Tim Jenness added a comment -

            Looks okay to me. Please remember to update the publishing system when you are done to make sure that we don't publish the testdata packages through eups distrib. I'll leave it in review state for Kian-Tat Lim.

            Show
            tjenness Tim Jenness added a comment - Looks okay to me. Please remember to update the publishing system when you are done to make sure that we don't publish the testdata packages through eups distrib . I'll leave it in review state for Kian-Tat Lim .
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            @timj https://github.com/lsst/lsstsw/pull/153 should prevent eups distrib packages being created for the testdata* products.

            Show
            jhoblitt Joshua Hoblitt added a comment - @timj https://github.com/lsst/lsstsw/pull/153 should prevent eups distrib packages being created for the testdata* products.
            Hide
            tjenness Tim Jenness added a comment -

            Yes, I know, but you have to actually install those files in the right place on the right machine for that to be true.

            Show
            tjenness Tim Jenness added a comment - Yes, I know, but you have to actually install those files in the right place on the right machine for that to be true.
            Hide
            jhoblitt Joshua Hoblitt added a comment -
            Show
            jhoblitt Joshua Hoblitt added a comment - Jenkins deploys lsstsw from master on each build. https://github.com/lsst-sqre/jenkins-dm-jobs/blob/master/jobs/run_rebuild.groovy#L25
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            Kian-Tat Lim Do you think you'll have time to give this a quick look over today? This is the last issue blocking the 13.0 release.

            Show
            jhoblitt Joshua Hoblitt added a comment - Kian-Tat Lim Do you think you'll have time to give this a quick look over today? This is the last issue blocking the 13.0 release.
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            I've gone ahead and added lsst_obs to lsst_distrib as this ticket was marked as a blocker for the 13.0 release. We can roll it back if any problems crop up tomorrow.

            Show
            jhoblitt Joshua Hoblitt added a comment - I've gone ahead and added lsst_obs to lsst_distrib as this ticket was marked as a blocker for the 13.0 release. We can roll it back if any problems crop up tomorrow.

              People

              Assignee:
              jhoblitt Joshua Hoblitt
              Reporter:
              tjenness Tim Jenness
              Reviewers:
              Kian-Tat Lim
              Watchers:
              Frossie Economou, John Parejko, John Swinbank, Joshua Hoblitt, Kian-Tat Lim, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.