Details
-
Type:
RFC
-
Status: Implemented
-
Resolution: Done
-
Component/s: DM
-
Labels:None
Description
This is a proposal to improve the DM Developer Guide's (developer.lsst.io) content organization.
The Developer Guide, in its modern form, is now over 2 years old. When we first set out to build the Developer Guide, we had a rough idea of what we wanted to put in it based on existing wiki pages. But since then, the Developer Guide has grown substantially (thanks to 28 contributors!). I'm finding more and more that it's becoming difficult to put new content into the existing topic hierarchy. To keep the Developer Guide healthy, I think it's time for us to invest in some content reorganization.
The core of this proposal is a new set of top-level topic sections:
- Team: This is for pages about the DM subsystem itself. Topics include the onboarding checklist, our culture and conduct standards, and subsites for individual teams (DRP has one, others can too).
- Communication: This is for pages that document how DM communicates. Here we'll talk about Slack, community.lsst.org, Confluence, RFCs, RFDs, meetings/conferences, and Zenodo.
- Project documentation: This is for pages that document how we create and maintain DM's project documents: change-controlled documents, papers, and technical notes. Note that topics about creating user documentation for software and services aren't in this section because I'm finding that user documentation is an intrinsically different activity in both an organizational and technical sense.
- Work management: This is for pages that describe how we get work done and track it. Here we'd have an overview topic of our development workflow, and detailed pages about JIRA and our agile practices.
- Coding: This is for pages that you'll refer to when you're working inside Data Management codebases. This section is split into subsections, where each subsection corresponds to a technology or product. For example, a C++ subsection, a Python subsection, and a Git/GitHub subsection. Within each subsection there will typically be a style guide with prescriptive guidelines for how you should/must use the technology, followed by descriptive pages meant to share knowledge and expertise (each topic page will be labeled as such). I also envision a "DM Stack" subsection where we can collect information specifically about working in our EUPS stack, such as packaging. DM's other products could get their own coding guides too.
- Developer infrastructure: This is for pages that document DM's hosted developer services. Currently this section of the site only has NCSA-hosted infrastructure (like lsst-dev), but I'd like to expand it to include documentation for our other services like Jenkins and even the prototype science platform deployments.
- Writing for users: This content doesn't exist yet, but this is where we'll have our user documentation style guide. These pages will help us write consistently for documentation sites like pipelines.lsst.io. My intention is to seed this with the Google Developer Documentation Style Guide. Documentation for how to technically contribute documentation for specific projects (like pipelines.lsst.io) will go into the corresponding Coding guide. Whether or not this section will eventually exist is the subject of other tickets/RFCs.
See the attached PDF that maps out these sections, along with specific topics that I expect to go in them.
Potential questions
- Will URLs change? Yes, I'm proposing changes to page URLs to put content into the new topic hierarchy. This is also an opportunity to give us consistent URL conventions, like [hyphens for connecting words].
- How do we mitigate the impact of URL changes? Whenever a page moves, I'll leave a redirect page where it originally existed. This will be done manually, though later on we intend to build a more automated approach for managing redirects in our Sphinx sites. I'll also search for links to developer.lsst.io pages in our other sites and documents and push updates when necessary.
- Will existing pages get split apart? In some cases, yes, I'd like to split and refactor existing topic pages.
- What is the timeline for this change? Once approved, I intend to implement these changes rapidly so that the new layout is available before DM's meeting in March.
- I don't think my Topic X will fit in that hierarchy. If you have a Developer Guide topic in mind and you don't think it will work within this proposed organization, let me know. Let's work together to get this right.
Attachments
Issue Links
Activity
Why is Git/GitHub under "Coding" and not "Work Management"? To my mind it's more like Jira – a tool to organize work as it gets done – than something related to working with code or Stack packages.
Interesting point. I’d have to think about that. I’ll say that I made JIRA stuff its own section because we use JIRA for many purposes, and I wanted to have a common point for all that information.
The boundaries between Team/Communication/Developer Infrastructure seem kind of blurry.
Could you provide some examples?
Where do the current C++/Python documentation guides go in this system? "Coding"?
In the guides of the specific technology, as described in the RFC text, and illustrated in the attached “developer-map.pdf”.
I'm a little wary of putting "Documenting C++", "Documenting Python", and "Writing Documentation for Users" in different top-level sections. I think it'd be better to have a single "How to document code" section (perhaps a subsection of "Writing Documentation for Users") that covers cross-language code concepts like classes and functions, with specializations for the different programming languages as the last level of branches in the tree. That may be a bigger reorganization than you had in mind, as it basically involves interleaving the existing Doxygen and NumpyDoc sections, but I think it's something to aim for even if we can't accomplish it initially.
Jim Bosch, thanks for the comment. Keep in mind that the Developer Guide is a website, not a book or PDF. This lets use links and topic-based organization. The top-level organization is really about that initial discovery experience on the front page.
I think I’d solve your issue by keeping the Doxygen format, Numpydoc, and JSDoc topics alongside the style guides for their respective languages because they really are extensions of the normative style guides. Then in the User documentation subsite there’s nothing stopping us from having a topic index page that links out to the individual pages on writing API references for each language we develop in. Basically we can have both organizations simultaneously.
I’m taking my cue here from Google, and most other tech companies, that create fairly universal English style guides that focus on usage issues, and let their internal guides for specific tools/products take care of application details.
The uploaded map is really nice to understand the structure. Where do I find the manuals for developers/users? Some examples: how to use Firefly JS API? How to use DAX API to do searches? May be they are not strictly only for developers. But some developers need them.
Thanks Xiuqin Wu [X] for bringing up that point. When we're documenting DM products, which includes APIs, we’ll have user guides specifically for those products.
pipelines.lsst.io is a current example, and we’ll certainly have ones for Firefly’s API, and DAX (see http://ls.st/ldm-294 §3.4 and Figures 6 and 7).
I do propose a “DM Stack” topic in developer.lsst.io, but that’s because there are some aspects of our EUPS stack that are only relevant to DM staff, like running stack-os-matrix test jobs, or packaging third party packages. But Science Pipelines APIs and user guides will all go in pipelines.lsst.io.
I’ll adopt this as I see consensus that the proposed organization will be better than what we currently have. The implementation ticket is DM-13656. I intend to do this this week so that I can concurrently ship some major documentation improvements related to writing user documentation for the DM Stack before the JTM starting on March 6.
The major concerns that have arisen from the comments are:
- Should JIRA be privileged with its own “work management” section as I proposed? Should GitHub be documented with it? Or should these follow our other technology-specific guides in a more flattened hierarchy.
- A reminder that user documentation weaves through multiple technologies and contexts, in addition to being its own discipline, so we must ensure that documentation on writing docs is properly cross linked and indexed.
However, I find a couple of things in the proposed structure confusing: