Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-733

Add Documenteer to the Conda environment (scipipe_conda_env)

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      I would like to add Documenteer, specifically the lsst-documenteer-pipelines meta-package (available on conda-forge), to the Conda environment for the stack (scipipe_conda_env). This change will provide all of our documentation dependencies in the default Python environment for both developers and CI systems. This change will also deprecate our use of a requirements.txt file in the pipelines_lsst_io repository for setting the version of Documenteer that should be used for CI and local builds of stack documentation.

      Concurrently, I would like to upgrade the version of Documenteer that we are using for production to 0.6.2 (from 0.5.6). As an immediate effect, this will upgrade us to the current versions of packages in our documentation ecosystem, such as Sphinx (1.7 to 3.2), Numpydoc, and sphinx-automodapi. This upgrade will also immediately improve the developer experience by solving a critical bug related to "recursion errors" in single-package documentation builds by changing how Documenteer uses the Sphinx configuration system. See the Documenteer release notes for details.

      Through this RFC, the final output of the documentation will not change in any outwardly-visible ways.

      Future benefits

      This proposal paves the way for future improvements (though they are not in the scope of this RFC's implementation):

      • Sphinx documentation builds could be run as part of the stack-os-matrix Jenkins job. Doing so would help us to validate docstrings and reStructuredText and even push draft documentation versions as part of ticket branch CI to improve the efficiency of documentation reviews.
      • Documenteer 0.6 includes the capability to configure and run a Doxygen build, embed that Doxygen content into the Sphinx documentation, and cross-reference from Sphinx documentation into the Doxygen-based C++ API reference. This capability is disabled in our current usage but can be turned on following a future RFC.
      • Take advantage of new Sphinx features (such as support for incorporating type annotations in Python API documentation).

      Tasks to implement this RFC

      1. Add lsst-documenteer-pipelines to scipipe_conda_env.
      2. Update the pipelines_lsst_io repository. First, change the conf.py file to use the new configuration API as outlined in https://documenteer.lsst.io/pipelines/configuration.html. Second, remove the requirements.txt file.
      3. Update the documenteer job in Jenkins so that it uses the Documenteer installed in the conda environment rather than installing an additional (and conflicting) copy of documenteer through pip.
      4. Update the conf.py files of packages (doc/conf.py) to use the new configuration API. We have endeavoured to maintain backwards compatibility in the new version of Documenteer with the old configuration API, but there's still a great benefit to upgrading packages as quickly and uniformly as possible.
      5. Update documentation (primarily in the Developer Guide) and templates to reflect the new conf.py files and the fact that Documenteer is now included in the Conda environment.

      Timing

      The 21.0.0 release of the LSST Science Pipelines is expected in November, and I am happy to work around this release, either by implementing the work now or delaying until after the next major release.

        Attachments

          Issue Links

            Activity

            Hide
            ktl Kian-Tat Lim added a comment - - edited

            While DM-26671 is about using rubin-env rather than specifically adding documenteer, documenteer has already been added to the environment in https://github.com/conda-forge/rubinenv-feedstock/pull/7. We'll therefore use DM-26671 as a triggered implementation ticket.

            If there are additional tickets for Jenkins updates etc., please link them.

            Show
            ktl Kian-Tat Lim added a comment - - edited While DM-26671 is about using rubin-env rather than specifically adding documenteer, documenteer has already been added to the environment in https://github.com/conda-forge/rubinenv-feedstock/pull/7 . We'll therefore use DM-26671 as a triggered implementation ticket. If there are additional tickets for Jenkins updates etc., please link them.
            Hide
            ktl Kian-Tat Lim added a comment -

            Apologies for not noticing that the addition to rubin-env was done incorrectly. (I looked at the title and not the content of the RFC.) This will be fixed in rubin-env 0.2.0.

            Show
            ktl Kian-Tat Lim added a comment - Apologies for not noticing that the addition to rubin-env was done incorrectly. (I looked at the title and not the content of the RFC.) This will be fixed in rubin-env 0.2.0.
            Hide
            ktl Kian-Tat Lim added a comment -

            Hm.  May have pushed the button prematurely.  While the title action has definitely been done (long ago), some of the follow-ups have not.  Nevertheless, I think they may belong in a different RFC (or not even require one).

            Show
            ktl Kian-Tat Lim added a comment - Hm.  May have pushed the button prematurely.  While the title action has definitely been done (long ago), some of the follow-ups have not.  Nevertheless, I think they may belong in a different RFC (or not even require one).
            Hide
            tjenness Tim Jenness added a comment -

            Do you want to remove DM-28328 from this RFC then?

            Show
            tjenness Tim Jenness added a comment - Do you want to remove DM-28328 from this RFC then?
            Hide
            ktl Kian-Tat Lim added a comment -

            I'll leave that for the owner (as I should have done for the overall RFC status).

            Show
            ktl Kian-Tat Lim added a comment - I'll leave that for the owner (as I should have done for the overall RFC status).

              People

              Assignee:
              jsick Jonathan Sick
              Reporter:
              jsick Jonathan Sick
              Watchers:
              Brian Van Klaveren, Gabriele Comoretto [X] (Inactive), John Parejko, Jonathan Sick, Kian-Tat Lim, Krzysztof Findeisen, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  Jenkins Builds

                  No builds found.