Fix Version/s: None
Component/s: Stack Documentation and UX
This is a hack day sprint to convert all remaining content on https://confluence.lsstcorp.org/display/LDMDG to reStructuredText content in the Sphinx project at https://github.com/lsst-sqre/dm_dev_guide and published at http://developer.lsst.io.
The top priority for this sprint is to port all content into reST and have it tracked by Git.
Sprint ground rules
- Before the sprint, clone https://github.com/lsst-sqre/dm_dev_guide.git and pip install -r requirements.txt in a Python 2.7 environment so that you can locally build the docs (make html).
- Claim a page from the list below by putting your name on it. Put a checkmark on the page when you’ve merged it to the ticket branch (see below).
- See http://developer.lsst.io/en/latest/docs/rst_styleguide.html for guidance on writing our style of reStructuredText. Pay attention to the heading hierarchy and labelling for internal links.
- If you use Pandoc to do an initial content conversion, you still need to go through the content line-by-line to standardize the reStructuredText. I personally recommend copy-and-pasting-and-formatting instead of using Pandoc.
- Your Git commit messages should include the URL of the original content from Confluence.
- Merge your work onto the tickets/
DM-5013ticket branch. Rebase your personal work branch before merging. JSick is responsible for merging this ticket branch to master.
- Put a note at the top of the confluence page with the new URL; root is http://developer.lsst.io/en/latest/.
Planned Developer Guide Table of Contents
We’re improving the organization of DM’s Developer Guide; there isn’t a 1:1 mapping of Confluence pages to developer.lsst.io pages. Below is a proposed section organization and page structure. These sections can still be refactored based on discussion during the hack day.
Getting Started — /getting-started/
- ✅ Onboarding Checklist (Confluence: Getting Started in DM). I’d like this to eventually be a quick checklist of things a new developer should do. It should be both a list of accounts the dev needs to have created, and a list of important developer guide pages to read next. The NCSA-specific material should be spun out. [Jonathan Sick]
- Communication Tools (new + DM Confluence Communication and Links). I see this as being an overview of what methods DM uses to communicate, and what method should be chosen for any circumstance.
- Finding Code on GitHub (new). This should point out all of the GitHub organizations that a developer might come across (DM and LSST-wide), and point out important repositories within each organization. Replaces the confluence page LSST Code Repositories
Processes — /processes/
- ✅ Team Culture and Conduct Standards (confluence)
- ✅ DM Development Workflow with Git, GitHub, JIRA and Jenkins (new & Confluence: git development guidelines for LSST + Git Commit Best Practices + DM Branching Policy)
- ✅ Discussion and Decision Making Process (new & confluence)
- ✅ DM Wiki Use (confluence) [John Swinbank]
- ✅ Policy on Updating Doxygen (confluence); needs to be addressed with TCT. Inter-link with the developer workflow page. [Jonathan Sick] (we’re just re-pointing the Confluence page to the workflow document)
- ✅ Transferring Code Between Packages (confluence) [John Swinbank]
Policy on Changing a Baseline Requirement(confluence)
- ✅ Project Planning for Software Development (confluence) [John Swinbank]
- ✅ JIRA Agile Usage (confluence) [John Swinbank]
Technical/Control Account Manager Guide(confluence) (Do not port; see discussion below.)
- Licensing (new) Need a centralized page to discuss license and copyright policies; include boilerplate statements.
Coding Guides — /coding/
- ✅ Introduction and note on stringency language (confluence: DM Coding Style Policy)
- ✅ DM Python Style Guide (confluence: Python Coding Standard)
- ✅ DM C++ Style Guide (confluence pages: C++ Coding Standard + C++ General Recommendations + C++ Naming Conventions + C++ Files + C++ Statements + C++ Layout and Comments + Policy on use of C++11/14 + On Using ‘Using’)
- Coding Style Linters (new; draft from confluence C++ Coding Standards Compliance and Python Coding Standards Compliance
- ✅ Using C++ Templates (confluence); this page needs to severely edited or re-written, however.
- ✅ Profiling (confluence). Also add a section ‘Using Valgrind with Python' (new) [Jonathan Sick]
- ✅ Boost Usage (TRAC) [Tim Jenness]
- ✅ Software Unit Test Policy (confluence) [John Swinbank]
- ✅ Unit Test Coverage Analysis (confluence) [John Swinbank]
- ✅ Unit Testing Private C++ Functions (trac) [John Swinbank]
Writing Docs — /docs/
- Introduction (new): Overview of DM’s documentation needs; links resources on technical writing.
- English Style Guide (new): Supplement the LSST Style Manual and provide English style guidance specific to DM. Capitalization of different heading levels; use of Chicago Manual of Style; a ‘this, not that’ table of spelling and word choices.
- ✅ ReStructuredText Style Guide (new)
- ✅ Documenting Stack Packages (new)
- ✅ Documenting Python Code (new)
- ✅ Documenting C++ Code (confluence, adapted from Documentation Standards); needs improvement
- ✅ Writing Technotes (new; port README from lsst-technote-bootstrap)
Developer Tools — /tools/
- ✅ Git Setup and Best Practices (new)
- ✅ Using Git Large File Storage (LFS) for Data Repositories (new)
- ✅ JIRA Work Management Recipes (new)
- ✅ Emacs Configuration (Confluence). See
DM-5045for issue with Emacs config repo - Jonathan Sick
- ✅ Vim Configuration (Confluence) - Jonathan Sick
Developer Services — /services/
- ✅ NCSA Nebula OpenStack Guide (Confluence: User Guide + Starting an Instance + Using Snapshots. Add the Vagrant instructions too from SQR-002? [Jonathan Sick]
- ✅ Using lsst-dev (Confluence: notes Getting Started + Developer Tools at NCSA
- ✅ Using the Bulk Transfer Server at NCSA (confluence) [Jonathan Sick]
Build, Test, Release — /build-ci/
- Eups for LSST Developers (new) [John Swinbank]
- ✅ The LSST Software Build Tool → ‘Using lsstsw and lsst-build' (confluence); lsstsw and lsst-build documentation. [John Swinbank]
- Using DM’s Jenkins for Continuous Integration (new) Frossie Economou
- ✅ Adding a New Package to the Build(confluence) [John Swinbank]
- ✅ Distributing Third-Party Packages with Eups (confluence) [John Swinbank]
- ✅ Triggering a Buildbot Build (confluence) Frossie Economou
- ✅ Buildbot Errors FAQ (confluence) Frossie Economou
- * Buildbot configuration (confluence Frossie Economou
- Creating a new DM Stack Release (confluence); though this page or a modern equivalent should probably belong with the software docs? Frossie Economou
A lot of work should go into this section. Have something about Scons? Or maybe that belongs in the doc of each relevant software product.
Leftover Confluence pages
The following pages should be moved to a separate Confluence space run by NCSA:
The following pages are either not relevant, generally misplaced, or need to be updated/recalibrated:
- Git Crash Course
- Basic Git Operations
- Handling Git Push Problems
- LSST Code Repositories; see the proposed “Finding Code on GitHub” page for a replacement.
- Standards and Policies: this is a good TOC for the Confluence docs; but not longer needed for the new docs.
- Documentation Guidelines. Some of this could be re-purposed into an intro to the ‘Writing Documentation’ section; some of this should go in a ‘Processes' page.
- DM Acknowledgements of Use: this probably belongs in documentation for the software projects that actually used this work.
- relates to
DM-4038 Update boost usage documentation
DM-5061 Provide documentation on EUPS for LSST Developers
DM-5095 Redirect confluence based pages to new developer guide.
DM-5060 Create an overview/flowchart doc of DM communication platforms
DM-5062 Write introduction to documentation writing
DM-5063 Page with links to DM documents and code resources for Developer Guide
DM-5064 Port Orchestration page from DM Confluence to developer.lsst.io
DM-5065 Port ‘Getting started with stack development’ page from DM’s Confluence
DM-5066 Port Doxygen Tips and Tricks from DM Confluence
DM-5067 Port ‘How to document a task’ from DM Confluence
Using the Redirection plugin (licensed but free) which provides a macro to forward from removed Confluence content to the new location. Eventually the pgaes can be removed completely.
Sprint is complete, all relevant pages from the DM Developer Guide on Confluence have been ported over. Tickets will be made for new developer guide additions and migration of relevant content in the DM confluence space as well as Trac.
What's the status on redirection? The few pages I checked on Confluence are not migrating to the new docs – should they all now be doing so or is that still a work in progress?
Frossie Economou and John Swinbank: I’m going to try rebasing tickets/
DM-5013onto the new master. Hold off on using the tickets/ DM-5013branch for now in case things get messy.