Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33898

OCPS calibrations script: add Watcher alarms/warnings when verification fails

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • None
    • None
    • 7
    • Data Release Production
    • No

    Attachments

      Issue Links

        Activity

          Yes it is the new event added in DM-38097 which will be part of ts_xml 16. It sounds to me as if your new rule should probably write those events instead of reporting traditional alarm severities. But I suggest talking to tribeiro about it.

          Regarding blocking calls to the butler: no Rule is allowed to hog the event loop, as that can render the Watcher unresponsive and unable to do its job. Doing anything with the butler (including creating it) must be done in a background thread unless you are 100% positive that it can never take more than a fraction of a millisecond (preferably much, much less). If I was writing the rule I would perform all butler operations in a background thread unless it is something intrinsically trivial and guaranteed to be very fast.

          As to "extra failures" I was just wondering if you had more than one level of severity in mind. But if you use the new event instead of alarm severities, it is not important.

          rowen Russell Owen added a comment - Yes it is the new event added in DM-38097 which will be part of ts_xml 16. It sounds to me as if your new rule should probably write those events instead of reporting traditional alarm severities. But I suggest talking to tribeiro about it. Regarding blocking calls to the butler: no Rule is allowed to hog the event loop, as that can render the Watcher unresponsive and unable to do its job. Doing anything with the butler (including creating it) must be done in a background thread unless you are 100% positive that it can never take more than a fraction of a millisecond (preferably much, much less). If I was writing the rule I would perform all butler operations in a background thread unless it is something intrinsically trivial and guaranteed to be very fast. As to "extra failures" I was just wondering if you had more than one level of severity in mind. But if you use the new event instead of alarm severities, it is not important.

          I agree that warnings are fine for now.  We'll may want to add a "critical" warning in the future when we go on-sky for real (which I would assume would require an operator response), but for now just running and ignoring any test failures is good enough.

          czw Christopher Waters added a comment - I agree that warnings are fine for now.  We'll may want to add a "critical" warning in the future when we go on-sky for real (which I would assume would require an operator response), but for now just running and ignoring any test failures is good enough.

          Regarding:

          > Yes it is the new event added in DM-38097 which will be part of ts_xml 16. It sounds to me as if your new rule should probably write those events instead of reporting traditional alarm severities. But I suggest talking to Tiago Ribeiro about it.

          Tiago says that he agrees with this.

          plazas Andrés Alejandro Plazas Malagón added a comment - Regarding: > Yes it is the new event added in DM-38097 which will be part of ts_xml 16. It sounds to me as if your new rule should probably write those events instead of reporting traditional alarm severities. But I suggest talking to Tiago Ribeiro about it. Tiago says that he agrees with this.

          This ticket has created the code for the alarm and a test:

          The test uses two auxiliary files with OCPS reults and cp_verify bias statistics

          plazas Andrés Alejandro Plazas Malagón added a comment - This ticket has created the code for the alarm and a test: Alarm: ts_watcher/python/lsst/ts/watcher/rules/cp_verify_alarm.py : cp_verify_alarm.py Test: ts_watcher/tests/rules/test_cp_verify_alarm.py : test_cp_verify_alarm.py The test uses two auxiliary files with OCPS reults and cp_verify bias statistics cp_verify bias statistics: ts_watcher/tests/rules/data/cp_verify_alarm/cp_verify_bias_stats.yaml : cp_verify_bias_stats.yaml OCPS bias call response: ts_watcher/tests/rules/data/cp_verify_alarm/ocps_bias_verify_response.yaml : ocps_bias_verify_response.yaml

          Ticket DM-38844 will implement and run any additional required unit tests (in addition to the test above), and finish the implementation and running of the alarm for ts_externalscripts/blob/develop/python/lsst/ts/externalscripts/base_make_calibrations.py.

          plazas Andrés Alejandro Plazas Malagón added a comment - - edited Ticket DM-38844 will implement and run any additional required unit tests (in addition to the test above), and finish the implementation and running of the alarm for ts_externalscripts/blob/develop/python/lsst/ts/externalscripts/base_make_calibrations.py .

          People

            plazas Andrés Alejandro Plazas Malagón
            plazas Andrés Alejandro Plazas Malagón
            Alysha Shugart, Andrés Alejandro Plazas Malagón, Christopher Waters, Russell Owen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Jenkins

                No builds found.