Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-6328

add hsc driver script to validate_drp

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: QA
    • Labels:

      Description

      Add an equivalent of examples/runChftTest.sh to validate_drp to process validation_data_hsc.

        Attachments

          Issue Links

            Activity

            Hide
            wmwood-vasey Michael Wood-Vasey added a comment -

            Oh. I didn't realize this. I understand now. You're saying ci_hsc doesn't install in the stack install, it just runs the tests.

            I've been directly running setup on a separate directory copy of the build directory; when doing that then CI_HSC_DIR gets defined. That's how I've been using it.

            I'll have to think about how to refer to these data if they don't get installed.

            Show
            wmwood-vasey Michael Wood-Vasey added a comment - Oh. I didn't realize this. I understand now. You're saying ci_hsc doesn't install in the stack install, it just runs the tests. I've been directly running setup on a separate directory copy of the build directory; when doing that then CI_HSC_DIR gets defined. That's how I've been using it. I'll have to think about how to refer to these data if they don't get installed.
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            My preference would be for ci_hsc to install like other products but we can clone it by hand if necessary.

            Show
            jhoblitt Joshua Hoblitt added a comment - My preference would be for ci_hsc to install like other products but we can clone it by hand if necessary.
            Hide
            wmwood-vasey Michael Wood-Vasey added a comment - - edited

            There's not a particular use case for runHscQuickTest.sh that's important. It's really only as a minor time savings to catch reasons runHscTest.sh might fail without having to download the 700 GB of validation_data_hsc.

            I will add a descriptive note in the README about having to explicitly set up the cloned repo of ci_hsc if one wants to use runHscQuickTest.sh and a reminder that ci_hsc doesn't install anything into the stack dir.

            The current ticket was meant to produce something that could be run, e.g., by SQuaSH. That's the full runHscTest.sh script.

            Show
            wmwood-vasey Michael Wood-Vasey added a comment - - edited There's not a particular use case for runHscQuickTest.sh that's important. It's really only as a minor time savings to catch reasons runHscTest.sh might fail without having to download the 700 GB of validation_data_hsc . I will add a descriptive note in the README about having to explicitly set up the cloned repo of ci_hsc if one wants to use runHscQuickTest.sh and a reminder that ci_hsc doesn't install anything into the stack dir. The current ticket was meant to produce something that could be run, e.g., by SQuaSH. That's the full runHscTest.sh script.
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            St. Michael Wood-Vasey has been very patient while we iterated through the ops model for running the `hsc` dataset. In the end, we determined that validation_data_hsc in too large and cumbersome to allow EUPS to manage it as a product dependency. It will be the responsibility of the end-user (or software agent) to manually clone/setup this the test dataset.

            Show
            jhoblitt Joshua Hoblitt added a comment - St. Michael Wood-Vasey has been very patient while we iterated through the ops model for running the `hsc` dataset. In the end, we determined that validation_data_hsc in too large and cumbersome to allow EUPS to manage it as a product dependency. It will be the responsibility of the end-user (or software agent) to manually clone/ setup this the test dataset.
            Hide
            wmwood-vasey Michael Wood-Vasey added a comment -

            Thanks very much to Joshua Hoblitt for the in-depth review that helped ensure that this work is ready for integration into regular processing.

            Show
            wmwood-vasey Michael Wood-Vasey added a comment - Thanks very much to Joshua Hoblitt for the in-depth review that helped ensure that this work is ready for integration into regular processing.

              People

              • Assignee:
                wmwood-vasey Michael Wood-Vasey
                Reporter:
                jhoblitt Joshua Hoblitt
                Reviewers:
                Joshua Hoblitt
                Watchers:
                Angelo Fausti, Frossie Economou, John Parejko, Jonathan Sick, Joshua Hoblitt, Michael Wood-Vasey
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel