# Update LDM-533 with latest test cases added in jira

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s: None
• Labels:
None
• Team:
Architecture

#### Description

Update LDM-533 with latest test cases added in jira.

This will update the entire document to use the new approach.  There will be some differences ion the layout but there should not be any difference in the content derived from the existing issue of the test specification.

#### Activity

Hide
Gabriele Comoretto added a comment -

This is the first time where we try to issue a test specification with:

• information imported in Jira and then "docugenerated"
• new test cases added directly in jira.

In this context I suggest the following:

1. review should first verify that the existing test cases are consistent and al information has been imported correctly;
2. review the new test cases (still in status draft) in order that they can be marked as "Approved" before the issue of this document.

Show
Gabriele Comoretto added a comment - This is the first time where we try to issue a test specification with: information imported in Jira and then "docugenerated" new test cases added directly in jira. In this context I suggest the following: review should first verify that the existing test cases are consistent and al information has been imported correctly; review the new test cases (still in status draft) in order that they can be marked as "Approved" before the issue of this document.
Hide
John Swinbank added a comment -

Overall, this looks ok. I've left a number of comments on the GitHub PR. To summarize:

• I'm unhappy that this introduces so much new terminology and notation without any attempt to explain it. For example, I don't think we should be issuing test specifications referring to “critical events” until LDM-503 is updated to describe what a critical event is.
• The way requirements are being referred to is confusing. Requirements in LSE-61 use names of the form “DMS-REQ-NNNN”. They don't use names of the form “LVV-MMM”, and then don't have “V-01” or other suffixes. Are you referring to some particular version of the requirement? What's this notation supposed to tell the reader?
• There are a number of minor formatting issues (spaces in the wrong place, etc). I've noted these on GitHub were I spotted them, but I've not tried to be exhaustive (I assume the generation of this text is automated, so what I've indicated should be enough to fix the scripts doing the generation).
• If test cases are now being referred to with names of the form “LVV-XYZ”, we should just use those names and get rid of all the “AG-12-34” stuff. Similarly, we should not keep a naming convention in this document but mark it as obsolete — if it's obsolete, get rid of it.

One thing that doesn't apply directly to a line on GitHub is that the document contains some broken references which need to be fixed:

 LaTeX Warning: Hyper reference ag-00-00' on page 7 undefined on input line 44. LaTeX Warning: Hyper reference ag-00-05' on page 7 undefined on input line 45. LaTeX Warning: Hyper reference ag-00-10' on page 7 undefined on input line 47. LaTeX Warning: Hyper reference ag-00-15' on page 7 undefined on input line 48. LaTeX Warning: Hyper reference ag-00-20' on page 7 undefined on input line 49. LaTeX Warning: Hyper reference ag-00-25' on page 7 undefined on input line 50. 

For completeness, I note that the work carried out is not in compliance with the ticket. That states

there should not be any difference in the content.

but in fact there is a bunch of new material here which has been added to Jira but wasn't formerly in the document. Obviously, the new material is good, but I think you should edit the ticket to reflect this.

All that said: I have not done a word-for-word comparison with the previous version of this document, but I am happy that the old material is all here and correct (if slightly less well presented than it used to be).

I'm not sure what the procedure for marking new test cases as “approved” is (do we have documentation on that?).

Show
John Swinbank added a comment - Overall, this looks ok. I've left a number of comments on the GitHub PR. To summarize: I'm unhappy that this introduces so much new terminology and notation without any attempt to explain it. For example, I don't think we should be issuing test specifications referring to “critical events” until LDM-503 is updated to describe what a critical event is. The way requirements are being referred to is confusing. Requirements in LSE-61 use names of the form “DMS-REQ-NNNN”. They don't use names of the form “LVV-MMM”, and then don't have “V-01” or other suffixes. Are you referring to some particular version of the requirement? What's this notation supposed to tell the reader? There are a number of minor formatting issues (spaces in the wrong place, etc). I've noted these on GitHub were I spotted them, but I've not tried to be exhaustive (I assume the generation of this text is automated, so what I've indicated should be enough to fix the scripts doing the generation). If test cases are now being referred to with names of the form “LVV-XYZ”, we should just use those names and get rid of all the “AG-12-34” stuff. Similarly, we should not keep a naming convention in this document but mark it as obsolete — if it's obsolete, get rid of it. One thing that doesn't apply directly to a line on GitHub is that the document contains some broken references which need to be fixed: LaTeX Warning: Hyper reference ag-00-00' on page 7 undefined on input line 44. LaTeX Warning: Hyper reference ag-00-05' on page 7 undefined on input line 45. LaTeX Warning: Hyper reference ag-00-10' on page 7 undefined on input line 47. LaTeX Warning: Hyper reference ag-00-15' on page 7 undefined on input line 48. LaTeX Warning: Hyper reference ag-00-20' on page 7 undefined on input line 49. LaTeX Warning: Hyper reference ag-00-25' on page 7 undefined on input line 50. For completeness, I note that the work carried out is not in compliance with the ticket. That states there should not be any difference in the content. but in fact there is a bunch of new material here which has been added to Jira but wasn't formerly in the document. Obviously, the new material is good, but I think you should edit the ticket to reflect this. All that said: I have not done a word-for-word comparison with the previous version of this document, but I am happy that the old material is all here and correct (if slightly less well presented than it used to be). I'm not sure what the procedure for marking new test cases as “approved” is (do we have documentation on that?).
Hide
Gabriele Comoretto added a comment -

Thank you John for all the useful comments, here and in the pull request. I think we may need some iteration in order to improve.

I agree that New Terminology need to be explained. In the Specific case of "critical event" we should probably avoid having it in out test spec.

In the test management approach, test cases refers to Verification Elements (REQ-ID-V0X) and not to requirements. Probably this need to be explained in the test spec somewhere.

The formatting will remain an issue, until the export script will not improve.

We can remove completely the old naming convention, since there are test cases that have already been executed. However new test cases no Draftm shall not contain the old naming since it is not usefull.

I will fix the reference problem.

There shall be no difference in the "old" content. As you confirm lather in your comment.

In terms of approval of the new content, I suppose that a RFC should be used.

Show
Gabriele Comoretto added a comment - Thank you John for all the useful comments, here and in the pull request. I think we may need some iteration in order to improve. I agree that New Terminology need to be explained. In the Specific case of "critical event" we should probably avoid having it in out test spec. In the test management approach, test cases refers to Verification Elements (REQ-ID-V0X) and not to requirements. Probably this need to be explained in the test spec somewhere. The formatting will remain an issue, until the export script will not improve. We can remove completely the old naming convention, since there are test cases that have already been executed. However new test cases no Draftm shall not contain the old naming since it is not usefull. I will fix the reference problem. There shall be no difference in the "old" content. As you confirm lather in your comment. In terms of approval of the new content, I suppose that a RFC should be used.
Hide
John Swinbank added a comment -

Thanks Gabriele! I made a few comments on the new material on Jira (basically just fixing typos) but otherwise I am happy for this to be merged to master.

However, I'd request that at the time you do that, you file another ticket to describe the updates that need to be made to LDM-503 (or other documentation, as appropriate) to provide a proper description of the terminology and procedures in use in this new approach to verification. That ticket doesn't have to be worked immediately (certainly, not before the review!), but we do need a clear and coherent summary of exactly what we're doing, and that doesn't seem to exist at the moment.

Show
John Swinbank added a comment - Thanks Gabriele! I made a few comments on the new material on Jira (basically just fixing typos) but otherwise I am happy for this to be merged to master. However, I'd request that at the time you do that, you file another ticket to describe the updates that need to be made to LDM-503 (or other documentation, as appropriate) to provide a proper description of the terminology and procedures in use in this new approach to verification. That ticket doesn't have to be worked immediately (certainly, not before the review!), but we do need a clear and coherent summary of exactly what we're doing, and that doesn't seem to exist at the moment.
Hide
Eric Bellm added a comment -

It's mostly fine, I think, but the alert distribution and alert generation test cases have been collapsed into a single test specification ("test plan", I think, in the new JIRA language.)

If you want I can provide some text for the Objectives section (which I will also add to https://jira.lsstcorp.org/secure/Tests.jspa#/testPlan/LVV-P1)

Show
Eric Bellm added a comment - It's mostly fine, I think, but the alert distribution and alert generation test cases have been collapsed into a single test specification ("test plan", I think, in the new JIRA language.) If you want I can provide some text for the Objectives section (which I will also add to https://jira.lsstcorp.org/secure/Tests.jspa#/testPlan/LVV-P1 )
Hide
Eric Bellm added a comment -

After discussion with Gabriele Comoretto, we agreed that we should simply incorporate the text "Objectives" from the original "AG" LDM-503-3 test specification up into the "Approach" section of this document, since we're bringing over the JIRA test cases but not the JIRA test plans. I'll just do the edits myself on this branch unless John Swinbank objects--it does change the content, at some level.

Show
Eric Bellm added a comment - After discussion with Gabriele Comoretto , we agreed that we should simply incorporate the text "Objectives" from the original "AG" LDM-503-3 test specification up into the "Approach" section of this document, since we're bringing over the JIRA test cases but not the JIRA test plans. I'll just do the edits myself on this branch unless John Swinbank objects--it does change the content, at some level.
Hide
John Swinbank added a comment -

I do not object.

Show
John Swinbank added a comment - I do not object.
Hide
Gabriele Comoretto added a comment -

Changes merged into master

Show
Gabriele Comoretto added a comment - Changes merged into master

#### People

Assignee:
Gabriele Comoretto
Reporter:
Gabriele Comoretto
Reviewers:
Eric Bellm, John Swinbank
Watchers:
Eric Bellm, Gabriele Comoretto, John Swinbank