Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15099

Update LDM-533 with latest test cases added in jira

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Team:
      Architecture

      Description

      Update LDM-533 with latest test cases added in jira.

       

      This will update the entire document to use the new approach.  There will be some differences ion the layout but there should not be any difference in the content derived from the existing issue of the test specification.

       

        Attachments

          Activity

          Hide
          gcomoretto Gabriele Comoretto added a comment -

          This is the first time where we try to issue a test specification with:

          • information imported in Jira and then "docugenerated"
          • new test cases added directly in jira.

          In this context I suggest the following:

          1. review should first verify that the existing test cases are consistent and al information has been imported correctly;
          2. review the new test cases (still in status draft) in order that they can be marked as "Approved" before the issue of this document.

           

          Show
          gcomoretto Gabriele Comoretto added a comment - This is the first time where we try to issue a test specification with: information imported in Jira and then "docugenerated" new test cases added directly in jira. In this context I suggest the following: review should first verify that the existing test cases are consistent and al information has been imported correctly; review the new test cases (still in status draft) in order that they can be marked as "Approved" before the issue of this document.  
          Hide
          swinbank John Swinbank added a comment -

          Overall, this looks ok. I've left a number of comments on the GitHub PR. To summarize:

          • I'm unhappy that this introduces so much new terminology and notation without any attempt to explain it. For example, I don't think we should be issuing test specifications referring to “critical events” until LDM-503 is updated to describe what a critical event is.
          • The way requirements are being referred to is confusing. Requirements in LSE-61 use names of the form “DMS-REQ-NNNN”. They don't use names of the form “LVV-MMM”, and then don't have “V-01” or other suffixes. Are you referring to some particular version of the requirement? What's this notation supposed to tell the reader?
          • There are a number of minor formatting issues (spaces in the wrong place, etc). I've noted these on GitHub were I spotted them, but I've not tried to be exhaustive (I assume the generation of this text is automated, so what I've indicated should be enough to fix the scripts doing the generation).
          • If test cases are now being referred to with names of the form “LVV-XYZ”, we should just use those names and get rid of all the “AG-12-34” stuff. Similarly, we should not keep a naming convention in this document but mark it as obsolete — if it's obsolete, get rid of it.

          One thing that doesn't apply directly to a line on GitHub is that the document contains some broken references which need to be fixed:

          LaTeX Warning: Hyper reference `ag-00-00' on page 7 undefined on input line 44.
          LaTeX Warning: Hyper reference `ag-00-05' on page 7 undefined on input line 45.
          LaTeX Warning: Hyper reference `ag-00-10' on page 7 undefined on input line 47.
          LaTeX Warning: Hyper reference `ag-00-15' on page 7 undefined on input line 48.
          LaTeX Warning: Hyper reference `ag-00-20' on page 7 undefined on input line 49.
          LaTeX Warning: Hyper reference `ag-00-25' on page 7 undefined on input line 50.
          

          For completeness, I note that the work carried out is not in compliance with the ticket. That states

          there should not be any difference in the content.

          but in fact there is a bunch of new material here which has been added to Jira but wasn't formerly in the document. Obviously, the new material is good, but I think you should edit the ticket to reflect this.

          All that said: I have not done a word-for-word comparison with the previous version of this document, but I am happy that the old material is all here and correct (if slightly less well presented than it used to be).

          I'm not sure what the procedure for marking new test cases as “approved” is (do we have documentation on that?).

          Show
          swinbank John Swinbank added a comment - Overall, this looks ok. I've left a number of comments on the GitHub PR. To summarize: I'm unhappy that this introduces so much new terminology and notation without any attempt to explain it. For example, I don't think we should be issuing test specifications referring to “critical events” until LDM-503 is updated to describe what a critical event is. The way requirements are being referred to is confusing. Requirements in LSE-61 use names of the form “DMS-REQ-NNNN”. They don't use names of the form “LVV-MMM”, and then don't have “V-01” or other suffixes. Are you referring to some particular version of the requirement? What's this notation supposed to tell the reader? There are a number of minor formatting issues (spaces in the wrong place, etc). I've noted these on GitHub were I spotted them, but I've not tried to be exhaustive (I assume the generation of this text is automated, so what I've indicated should be enough to fix the scripts doing the generation). If test cases are now being referred to with names of the form “LVV-XYZ”, we should just use those names and get rid of all the “AG-12-34” stuff. Similarly, we should not keep a naming convention in this document but mark it as obsolete — if it's obsolete, get rid of it. One thing that doesn't apply directly to a line on GitHub is that the document contains some broken references which need to be fixed: LaTeX Warning: Hyper reference `ag-00-00' on page 7 undefined on input line 44. LaTeX Warning: Hyper reference `ag-00-05' on page 7 undefined on input line 45. LaTeX Warning: Hyper reference `ag-00-10' on page 7 undefined on input line 47. LaTeX Warning: Hyper reference `ag-00-15' on page 7 undefined on input line 48. LaTeX Warning: Hyper reference `ag-00-20' on page 7 undefined on input line 49. LaTeX Warning: Hyper reference `ag-00-25' on page 7 undefined on input line 50. For completeness, I note that the work carried out is not in compliance with the ticket. That states there should not be any difference in the content. but in fact there is a bunch of new material here which has been added to Jira but wasn't formerly in the document. Obviously, the new material is good, but I think you should edit the ticket to reflect this. All that said: I have not done a word-for-word comparison with the previous version of this document, but I am happy that the old material is all here and correct (if slightly less well presented than it used to be). I'm not sure what the procedure for marking new test cases as “approved” is (do we have documentation on that?).
          Hide
          gcomoretto Gabriele Comoretto added a comment -

          Thank you John for all the useful comments, here and in the pull request. I think we may need some iteration in order to improve.

          I agree that New Terminology need to be explained. In the Specific case of "critical event" we should probably avoid having it in out test spec.

          In the test management approach, test cases refers to Verification Elements (REQ-ID-V0X) and not to requirements. Probably this need to be explained in the test spec somewhere.

          The formatting will remain an issue, until the export script will not improve.

          We can remove completely the old naming convention, since there are test cases that have already been executed. However new test cases no Draftm shall not contain the old naming since it is not usefull.

          I will fix the reference problem.

          There shall be no difference in the "old" content. As you confirm lather in your comment.

          In terms of approval of the new content, I suppose that a RFC should be used.

          Show
          gcomoretto Gabriele Comoretto added a comment - Thank you John for all the useful comments, here and in the pull request. I think we may need some iteration in order to improve. I agree that New Terminology need to be explained. In the Specific case of "critical event" we should probably avoid having it in out test spec. In the test management approach, test cases refers to Verification Elements (REQ-ID-V0X) and not to requirements. Probably this need to be explained in the test spec somewhere. The formatting will remain an issue, until the export script will not improve. We can remove completely the old naming convention, since there are test cases that have already been executed. However new test cases no Draftm shall not contain the old naming since it is not usefull. I will fix the reference problem. There shall be no difference in the "old" content. As you confirm lather in your comment. In terms of approval of the new content, I suppose that a RFC should be used.
          Hide
          swinbank John Swinbank added a comment -

          Thanks Gabriele! I made a few comments on the new material on Jira (basically just fixing typos) but otherwise I am happy for this to be merged to master.

          However, I'd request that at the time you do that, you file another ticket to describe the updates that need to be made to LDM-503 (or other documentation, as appropriate) to provide a proper description of the terminology and procedures in use in this new approach to verification. That ticket doesn't have to be worked immediately (certainly, not before the review!), but we do need a clear and coherent summary of exactly what we're doing, and that doesn't seem to exist at the moment.

          Show
          swinbank John Swinbank added a comment - Thanks Gabriele! I made a few comments on the new material on Jira (basically just fixing typos) but otherwise I am happy for this to be merged to master. However, I'd request that at the time you do that, you file another ticket to describe the updates that need to be made to LDM-503 (or other documentation, as appropriate) to provide a proper description of the terminology and procedures in use in this new approach to verification. That ticket doesn't have to be worked immediately (certainly, not before the review!), but we do need a clear and coherent summary of exactly what we're doing, and that doesn't seem to exist at the moment.
          Hide
          ebellm Eric Bellm added a comment -

          It's mostly fine, I think, but the alert distribution and alert generation test cases have been collapsed into a single test specification ("test plan", I think, in the new JIRA language.)

          If you want I can provide some text for the Objectives section (which I will also add to https://jira.lsstcorp.org/secure/Tests.jspa#/testPlan/LVV-P1)

          Show
          ebellm Eric Bellm added a comment - It's mostly fine, I think, but the alert distribution and alert generation test cases have been collapsed into a single test specification ("test plan", I think, in the new JIRA language.) If you want I can provide some text for the Objectives section (which I will also add to https://jira.lsstcorp.org/secure/Tests.jspa#/testPlan/LVV-P1 )
          Hide
          ebellm Eric Bellm added a comment -

          After discussion with Gabriele Comoretto, we agreed that we should simply incorporate the text "Objectives" from the original "AG" LDM-503-3 test specification up into the "Approach" section of this document, since we're bringing over the JIRA test cases but not the JIRA test plans. I'll just do the edits myself on this branch unless John Swinbank objects--it does change the content, at some level.

          Show
          ebellm Eric Bellm added a comment - After discussion with Gabriele Comoretto , we agreed that we should simply incorporate the text "Objectives" from the original "AG" LDM-503-3 test specification up into the "Approach" section of this document, since we're bringing over the JIRA test cases but not the JIRA test plans. I'll just do the edits myself on this branch unless John Swinbank objects--it does change the content, at some level.
          Hide
          swinbank John Swinbank added a comment -

          I do not object.

          Show
          swinbank John Swinbank added a comment - I do not object.
          Hide
          gcomoretto Gabriele Comoretto added a comment -

          Changes merged into master

          Show
          gcomoretto Gabriele Comoretto added a comment - Changes merged into master

            People

            Assignee:
            gcomoretto Gabriele Comoretto
            Reporter:
            gcomoretto Gabriele Comoretto
            Reviewers:
            Eric Bellm, John Swinbank
            Watchers:
            Eric Bellm, Gabriele Comoretto, John Swinbank
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: