Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-2190

Documentation for data loader

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • Qserv
    • None

    Description

      Vaikunth had some "expected" troubles playing with data loader options for his DM-1570 ticket. Main issue I believe is the absence of the documented use cases and their corresponding data loader options. I'll try to add a bunch of common use cases to RST documentation and also verify that all options behave as expected.

      Attachments

        Issue Links

          Activity

            Vaikunth, this is a follow-up to correct some troubles that you had with DM-1570, would be nice if you could find time to review it. The biggest change is a documentation for data loader, I think you are best qualified to review that and see if it could have helped you earlier To view documentation just do "scons doc" and "firefox doc/build/html/index.html" after checking out branch (or just go to https://github.com/lsst/qserv/blob/u/salnikov/DM-2190/doc/source/admin/data-loader.rst). There are also few code changes to support use cases which were broken before.

            salnikov Andy Salnikov added a comment - Vaikunth, this is a follow-up to correct some troubles that you had with DM-1570 , would be nice if you could find time to review it. The biggest change is a documentation for data loader, I think you are best qualified to review that and see if it could have helped you earlier To view documentation just do "scons doc" and "firefox doc/build/html/index.html" after checking out branch (or just go to https://github.com/lsst/qserv/blob/u/salnikov/DM-2190/doc/source/admin/data-loader.rst ). There are also few code changes to support use cases which were broken before.

            Hi Andy,

            Thanks for the thorough documentation. I have no comments since it seems to cover everything, including all cases that I had run into problems with. I hope all integration tests still pass with the changes to the loader?

            vaikunth Vaikunth Thukral added a comment - Hi Andy, Thanks for the thorough documentation. I have no comments since it seems to cover everything, including all cases that I had run into problems with. I hope all integration tests still pass with the changes to the loader?

            Thanks Vaikunth! Integration tests still work of course (for me at least). Now that all use cases are documented and work as they should you may want to revisit code that you added to integration tests, maybe something could be simplified there.

            Andy

            salnikov Andy Salnikov added a comment - Thanks Vaikunth! Integration tests still work of course (for me at least). Now that all use cases are documented and work as they should you may want to revisit code that you added to integration tests, maybe something could be simplified there. Andy

            Yep, that's why I was asking - I likely and inadvertently used a less-than-ideal or intended method to get it to work initially, probably worth revisiting and optimizing the test cases now.

            vaikunth Vaikunth Thukral added a comment - Yep, that's why I was asking - I likely and inadvertently used a less-than-ideal or intended method to get it to work initially, probably worth revisiting and optimizing the test cases now.

            People

              salnikov Andy Salnikov
              salnikov Andy Salnikov
              Vaikunth Thukral
              Andy Salnikov, Vaikunth Thukral
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.