I am not part of the CE team, but I have done user support in one way or another all my adult life and manage one of the most user-intensive services at Rubin so I ask for your indulgence to make some comments, bearing in mind that is is ultimately Leanne's call as it is her department.

A successful endeavour requires the optimal application of resources. Optimal does not mean best for every single individual (as that is usually as possible as perpetual motion), but best bang for your buck, and the buck here is the attention economy of the project staff. To give a non-computing example: if you are teaching an 800-student strong Astronomy 101 course, you are not telling your students "drop by any time if you ever have a question". That would be best for the student, but it is not best for you, and you are over-leveraged, which is why you have sign-ups for office hours.

So while for any individual it would be best if they can drop into our Slack and get the full attention of several people, with 8,000 users and a handful of developers that is an attention economy dumpsterfire. There is a reason why you don't get support from Amazon, or your congressperson or even your doctor on an interactive chat system.

Not only is chat a denial of service attack to highly leveraged staff but it is actually terrible for knowledge transfer. Anybody who has watched the endless goldfish bowl that is Twitch chat can vouch to that A user getting their questions answered does nothing to help the user 3 months later with the same question. Yes, Meredith's point that this helps support staff know what questions are asking is valid, but the same thing would happen if they had asked it on a web forum.

Now academia has a great tradition of self-organising support and collaboration communities (exhibit A, science collaborations). When we at SQuaRE first made a community web forum (colloquially CLO) platform available, one of its two express purposes was to divert support to a googlable forum, so the next user could find the answer without even asking. But note that in those days chat platforms like Slack had a very locked in organisational model and we also wanted the one strong technical advantage of chat: when you do need to engage a user in a tight troubleshooting loop ("what happens when you type this? what about that?") Slack really is better for everyone.

Meanwhile, the world has evolved. Slack has made it very easy to make and to belong to multiple workspaces (something that was impossible in the beginning IIRC), to invite guests into channels and so on. Moreover, as other parts of DM have come along, more of us are deeply involved in systems with no front-facing external user interactions and have larger codebases to support. We are not Amazon or Facebook, but we do have the need to scale support to a level where some of their user support modalities are more appropriate to us in some cases.

Finally, and I hope I don't make any enemies with this one, I include my own systems in them: "boutique" level support is a crutch. Helping someone on Slack at 10pm makes me feel good and like I did something useful and I go to bed happy with the praise of the grateful user, but what I really should have done is made sure the code was not flakey or the class design confusing, or the documentation lacking in the first place. I feel good when I should have felt bad.

I understand the current system seems to work well, and I am happy people are getting help on #newbies, but consider also the severe downsides to the current model:

• There are many more private channels than there used to before we added external users
• We cannot afford a number of useful operational Slack plugins as their cost model is to charge per number of active workspace users (1800 at the current time)
• I have already given up at even trying to follow the channels I used to because there's so much traffic - I have been trained to ignore unread marks in order to survive.

Okay so if anybody made it past these ramblings:

• I think the CE team is right to build support process around public, searchable, persistent platforms and to foster a peer-to-peer community around them.
• Slack gives us the ability to pull in a user who is particularly struggling as a guest in our Slack if we need to.
• Slack can be a valuable tool for individual communities to form around, but they don't need us to do that, they can form their own workspaces if they need to
• If there are specific areas where there is specific benefit to specific project staff from being immersed in the stream and being hyper available (eg science discussions across collaborations) then they should be free to engage in their own volition
• First-line support contact should never happen on Slack directly to developers in Operations, that way lies madness.

FWIW, if I was dictator of the earth, I would

• Leave the LSSTC Slack as a platform for collaborations and any project staff who wanted to engage with them (sounds like there's a lot of interest in some science staff for that), "I am a stack dev AMA" drop-ins etc
• Make a new staff-only project workspace for Rubin Ops so we can install bots, plugins, notifications that we can pay high attention to, exchange operational information in open rooms, etc. (and I don't mean NOIRLab either, that would also be too large)
• Support the CE team in creating frameworks to enable peer to peer support (that I can do without being dictator of the earth at least!)

I hope folks don't mind me adding my two cents here based on this experience with the ticketing system I built for central services at Stanford and experiences at Dropbox with Slack-based support and in Debian with separation of user and developer support.

I'll second Frossie's observations about the difficulties scaling chat-based support to a larger user base and add in another problem that plagued Slack at Dropbox: Lack of tracking. In order to provide a good support experience, we will want to be able to see which user questions have gone unanswered. This is difficult in Slack because of the free-form and untagged nature of the discussion. It's easy for someone to ask a question in the middle of another discussion and have it be lost. Either they'll get discouraged and go away, or they'll ping again and add yet more noise. To work around that, teams at Dropbox created elaborate support structures in Slack with multiple channels, a requirement to use threads, and even bots that looked for reaction emoji to try to track the status of questions, none of which worked well. By comparison, checking for whether something has been answered is a built-in feature of forums.

That was the key problem we had to solve at Stanford when I wrote our early ticketing system to replace an email support list. We had fairly elaborate processes (that we had to train each new help desk person on) for searching for unanswered questions, and for a while I was generating an unanswered questions list from tags in my own email client. We replaced that with a simple system for tracking whether a request had been answered and it hugely improved the consistency of our user support. Slack doesn't offer good features for this.

My experience in Debian is that if user support is mixed in with developer conversations, the very people who have the most expertise withdraw from both the support and the conversations and go off into private channels they perceive as less noisy, which makes communication worse for the project as a whole. If the user support can be kept in its own area and it's easy to see what has and hasn't been answered, even developers who mostly don't want to do user support will occasionally drop in and answer a few questions they find interesting.

All that leads me to support a model where the primary user interface for support is in a community forum with the features required to track unhandled requests and which encourages users to look for similar questions before adding new support load, keep a project staff Slack strictly separate, and provide some sort of facility to move specific user problems to a Slack other than the project staff Slack when they would benefit from the tighter communication loop.

i appreciate and agree with much of the experience of Frossie and Russ.

As a tiny point, though, I'd like to say that "not in Google" hardly means "does not exist" or "is not searchable".  In fact, I have found Slack's search to be invaluable at finding useful information, even more so after its improvement in 2018.  I may have good keyword and label selection skills (and patience at going through multiple result screens, not depending only on the first hit), but those can be trained relatively easily.

Finally getting to this one, as I missed it initially and John Swinbank reminded me of it when I asked him "Can I please direct someone to our support model so they stop sending me private Slack messages asking for help?" Thus demonstrating a definite advantage of a forum-like system (like Community or Jira): you can trivially get back into a conversation later on.

I'll note that there is a good "separation of resources" reason most medium to large projects have separate -user and -dev email lists. Those are overlapping but different groups of people, who have quite different needs. Similarly for -user and -operations, as will be our case in a couple of years.

I'll also second the change of CLO to Community, which I see has already been done in the proposal. CLO could also stand for confluence.lsstcorp.org and we have plenty of other acronyms.

Both the DM-SST and the CET will rely on existing documentation to generate answers when possible

It would be good to formalize a process for when that documentation is insufficient. Members of the CET should file tickets including suggested documentation wording, when they identify aspects of the docs that are lacking.

Given all that, I think I support this proposal as written. LSST Slack has already become too much for me to really follow daily, with the small handful of science collaborations I'm in. Add in several times as many people and I definitely would not want to become a CET. Supporting through Community is something I think I might enjoy doing, though. I wonder if that would be worth a quick vote in #dm-science-pipelines then? "Would you enjoy being a CET per the RFC-703 definition if support was primarily on 1) Slack 2) Community or 3) I do not want to be a CET?"

Thank you all for this excellent feedback and input. The outcome is summarized in DMTN-155.