Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-468

Accept the Science Platform Test Specification, LDM-540, as a baseline

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM, TCT
    • Labels:

      Description

      A version of the Science Platform Test Specification, LDM-540, is ready for submission to the DM change control process.

      It is available for now at https://ldm-540.lsst.io/v/DM-13864/index.html (I'll create an RFC branch in the repository shortly).

      This is the version written under DM-13397 in February 2018, with one small edit to add missing requirement IDs to a test case.

        Attachments

          Issue Links

            Activity

            Hide
            womullan Wil O'Mullane added a comment -

            Brian Van Klaveren Gabriele Comoretto  please have a look at the test spec

            Show
            womullan Wil O'Mullane added a comment - Brian Van Klaveren Gabriele Comoretto   please have a look at the test spec
            Hide
            gcomoretto Gabriele Comoretto added a comment -

            I had a look to the document and in principle it seems fine. I can' t say if the test cases are addressing correctly the corresponding requirements verification, but they seem well structured.

             

            I have a couple of minor comments:

            • on section 1.1 where Test Design are mentioned, but then I don't see them in the document, unless I have missed some part. We can always say that test designs are not mandatory, since they should just group test cases.
            • on section 2.6 it is not explained the test naming convention

             

            I think that all this information could be maintained in Jira or confluence, and then exported to a doc when ready, this make it easier to maintain an to get to the the final VCD.

             

            I see that scripts are used in some case to run the tests. These should be baselined, tagged maybe, in order to have a fix reference, in case there will be multiple runs on different versions of the scripts. 

            Show
            gcomoretto Gabriele Comoretto added a comment - I had a look to the document and in principle it seems fine. I can' t say if the test cases are addressing correctly the corresponding requirements verification, but they seem well structured.   I have a couple of minor comments: on section 1.1 where Test Design are mentioned, but then I don't see them in the document, unless I have missed some part. We can always say that test designs are not mandatory, since they should just group test cases. on section 2.6 it is not explained the test naming convention   I think that all this information could be maintained in Jira or confluence, and then exported to a doc when ready, this make it easier to maintain an to get to the the final VCD.   I see that scripts are used in some case to run the tests. These should be baselined, tagged maybe, in order to have a fix reference, in case there will be multiple runs on different versions of the scripts. 
            Hide
            gpdf Gregory Dubois-Felsmann added a comment -

            Thank you for your comments, Gabriele Comoretto.

            1. Re: "test designs" - As you know, we are relying on templates Wil brought over from Gaia. Not all the terminology was given a specific design in the LSST context. I confess I just left this sentence in place without questioning exactly how "test design" differed from the elaboration of test cases I provided. Can we just delete the words "test designs", unless there is a specific meaning agreed for this term in the LSST context?
            2. Re: section 2.6 - My fault, that was an editing oversight which I will fix.
            3. Re: maintaining information in Jira/Confluence - what information specifically?
            4. Re: version control on test scripts - in the test procedure I wrote "Clone the Github lsst/LDM-540 package. Record the SHA1 for the version of the package to be used. (Additional procedures, e.g., tagging, are still to be confirmed.)" because it was clear that a baseline needed to be recorded. I agree we should have tagging, but I didn't want to invent a tagging procedure just for LDM-540; can we get a central recommendation which we can then reference in all test specifications? Perhaps this is something Gabriele Comoretto and Brian Van Klaveren can collaborate on drawing up? As this process gets more mature and the tests are defined farther in advance we will probably also want to require an RFC to confirm the versions of test scripts to be used, at least for key milestones.
            Show
            gpdf Gregory Dubois-Felsmann added a comment - Thank you for your comments, Gabriele Comoretto . Re: "test designs" - As you know, we are relying on templates Wil brought over from Gaia. Not all the terminology was given a specific design in the LSST context. I confess I just left this sentence in place without questioning exactly how "test design" differed from the elaboration of test cases I provided. Can we just delete the words "test designs", unless there is a specific meaning agreed for this term in the LSST context? Re: section 2.6 - My fault, that was an editing oversight which I will fix. Re: maintaining information in Jira/Confluence - what information specifically? Re: version control on test scripts - in the test procedure I wrote "Clone the Github lsst/LDM-540 package. Record the SHA1 for the version of the package to be used. (Additional procedures, e.g., tagging, are still to be confirmed.)" because it was clear that a baseline needed to be recorded. I agree we should have tagging, but I didn't want to invent a tagging procedure just for LDM-540; can we get a central recommendation which we can then reference in all test specifications? Perhaps this is something Gabriele Comoretto and Brian Van Klaveren can collaborate on drawing up? As this process gets more mature and the tests are defined farther in advance we will probably also want to require an RFC to confirm the versions of test scripts to be used, at least for key milestones.
            Hide
            womullan Wil O'Mullane added a comment -

            please publish an updated document

             

            Show
            womullan Wil O'Mullane added a comment - please publish an updated document  
            Hide
            gpdf Gregory Dubois-Felsmann added a comment -

            The RFC-468 branch contains a version that deals with items 1 and 2 above.  I think our conclusion was that 3. and 4. did not require immediate action?

            Show
            gpdf Gregory Dubois-Felsmann added a comment - The RFC-468 branch contains a version that deals with items 1 and 2 above.  I think our conclusion was that 3. and 4. did not require immediate action?
            Hide
            womullan Wil O'Mullane added a comment -

            yes lets go with this version so we have it to go with the report.

            Show
            womullan Wil O'Mullane added a comment - yes lets go with this version so we have it to go with the report.
            Hide
            gpdf Gregory Dubois-Felsmann added a comment -

            Fully implemented: v1.0 of LDM-540 created and recorded as a baseline in Docushare according to the developer.lsst.io procedures.

            Show
            gpdf Gregory Dubois-Felsmann added a comment - Fully implemented: v1.0 of LDM-540 created and recorded as a baseline in Docushare according to the developer.lsst.io procedures.

              People

              • Assignee:
                gpdf Gregory Dubois-Felsmann
                Reporter:
                gpdf Gregory Dubois-Felsmann
                Watchers:
                Gabriele Comoretto, Gregory Dubois-Felsmann, John Swinbank, Leanne Guy, Tim Jenness, Wil O'Mullane
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Planned End:

                  Summary Panel