# Propose DM text for LCR-786 - elimination of Base raw data network outage buffer

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Team:
Architecture

## Description

Following up on a discussion at the project SE-IPT meeting today, the action here is to propose specific changes to the OSS and DMSR that capture the intent of the change in LCR-786, the elimination of the Base network outage buffer and the increase in size of the Camera DAQ data buffer.

Proposed language is meant to be introduced at the Systems Engineering meeting on January 30th, 2018.

## Attachments

 1 Confirm that the full-size NCSA DAQ test stand will be upgraded to 1TB devices along with the Camera DAQ itself Done Wil O'Mullane

## Activity

Hide
Tim Jenness added a comment -

LCR-786 is vague concerning its effect on DM (other than the addition of $40k to DM budget KLM20602A.PP to pay for the summit disks). The vote result stated: DM will remove the Base "network outage buffer" from the design of the data-handling system; Previously 14 days was mentioned as being removed from the base and that suggests we have to do something about DMS-REQ-0177: Base Facility Temporary Storage Specification: The Base Facility shall provide at least a time minBaseDataStorage (or half of that amount, redundantly) of raw data storage in the event of Base to Archive Center network outage. where minBaseDataStorage is 14 days, and that requirement is associated with DMS-REQ-0164 (Temporary Storage for Communications Links) and OSS-REQ-0054 (Base Data Buffer): Specification: The LSST system shall provide a limited buffer for data (raw, processed, and engineering) en route from the Base to the Archive Center, to allow science operations to proceed in the event of communications loss between the Base and Archive. The system shall have sufficient capacity to store all en route data for at least the design outage time, baseConnectivityLossTime [48 hours]. It was noted by Kian-Tat Lim that DMS-REQ-0177 might still be needed because some sort of buffer will be used for other parts of the system (but 0177 doesn't really say what the size should be). Which design documents referred to in the LCR discuss the base buffer? Are those all now fixed as part of the re-plan? Show Tim Jenness added a comment - LCR-786 is vague concerning its effect on DM (other than the addition of$40k to DM budget KLM20602A.PP to pay for the summit disks). The vote result stated: DM will remove the Base "network outage buffer" from the design of the data-handling system; Previously 14 days was mentioned as being removed from the base and that suggests we have to do something about DMS-REQ-0177: Base Facility Temporary Storage Specification: The Base Facility shall provide at least a time minBaseDataStorage (or half of that amount, redundantly) of raw data storage in the event of Base to Archive Center network outage. where minBaseDataStorage is 14 days, and that requirement is associated with DMS-REQ-0164 (Temporary Storage for Communications Links) and OSS-REQ-0054 (Base Data Buffer): Specification: The LSST system shall provide a limited buffer for data (raw, processed, and engineering) en route from the Base to the Archive Center, to allow science operations to proceed in the event of communications loss between the Base and Archive. The system shall have sufficient capacity to store all en route data for at least the design outage time, baseConnectivityLossTime [48 hours] . It was noted by Kian-Tat Lim that DMS-REQ-0177 might still be needed because some sort of buffer will be used for other parts of the system (but 0177 doesn't really say what the size should be). Which design documents referred to in the LCR discuss the base buffer? Are those all now fixed as part of the re-plan?
Hide
Tim Jenness added a comment - - edited

After discussion with Kian-Tat Lim on Slack we think that a reasonable approach is:

1. Remove DMS-REQ-0177
2. DMS-REQ-0164 be adjusted to end with "at or before the source end of that link."
3. DMS-REQ-0318: Changing 3-day in discussion section to summit buffer size + 1 day.
Show
Tim Jenness added a comment - - edited After discussion with Kian-Tat Lim on Slack we think that a reasonable approach is: Remove DMS-REQ-0177 DMS-REQ-0164 be adjusted to end with "at or before the source end of that link." DMS-REQ-0318: Changing 3-day in discussion section to summit buffer size + 1 day.
Hide
Kian-Tat Lim added a comment -

For the OSS, my proposal is:

1. OSS-REQ-0004: change "reprocessing" to "processing" (leftover from AP move).
2. OSS-REQ-0052: summitBufferTransferTime should be rewritten as a transfer rate (2 days per day).  Note that DMS-REQ-0165 catches up processing at one day per day, but transfer can and should be faster.
3. OSS-REQ-0055: may be better to rewrite this as a rate as well, although acceptable as is
4. Remove OSS-REQ-0054.
Show
Kian-Tat Lim added a comment - For the OSS, my proposal is: OSS-REQ-0004: change "reprocessing" to "processing" (leftover from AP move). OSS-REQ-0052: summitBufferTransferTime should be rewritten as a transfer rate (2 days per day).  Note that DMS-REQ-0165 catches up processing at one day per day, but transfer can and should be faster. OSS-REQ-0055: may be better to rewrite this as a rate as well, although acceptable as is Remove OSS-REQ-0054.
Hide
Tim Jenness added a comment -

Marking as done since a new LCR has been proposed: https://project.lsst.org/groups/ccb/node/2238

Show
Tim Jenness added a comment - Marking as done since a new LCR has been proposed: https://project.lsst.org/groups/ccb/node/2238

## People

• Assignee:
Tim Jenness
Reporter:
Gregory Dubois-Felsmann
Watchers:
Gregory Dubois-Felsmann, Jeff Kantor, Kian-Tat Lim, Tim Jenness