Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-1338

Make QSERV_RUN_DIR scripts able to detect qserv install paths using eups

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Qserv
    • Labels:
      None

      Description

      Ticket related to Mario email (subject: [QSERV-L] Some points/actions from the discussion today) :

      Making your "qserv data" directory independent of where qserv is installed

      I think this is a big one, and largely independent of EUPS. You have a problem where you want to use one set of test data potentially with different qserv binaries (not at the same time, of course). I'd argue you should refactor the scripts generated by qserv-configure to either:

      • get all their information about various paths from a single file, for example, from etc/paths.cfg.sh. Then you can easily regenerate just that file when you need to switch to a different qserv (or zookeper, or what not), or...
      • refactor the generated scripts to learn from the environment which binaries to run. I.e., if $QSERV_DIR is defined, use that qserv, etc. This will let you switch binaries by simply setup-ing the new one with EUPS.

      The two are not mutually exclusive – e.g., all of this logic could be in etc/paths.cfg.sh, and depending on whether this is a development build or a "non-EUPS" build, it can either pick up the paths from the environment or hardcode them.

      Assuming you did that, your development loop may look something like this:

          # assuming that qserv-configure.py has already been run
          # in ../qserv-run
       
          # do something with qserv-a clone
          cd qserv-a
          setup -r .
          ... do some edits ...
          scons
       
          # now do the tests
          cd ../qserv-run
          ./bin/qserv-start.sh
          ... do tests ...
          ./bin/qserv-stop.sh
       
          # now switch to qserv-b clone
          cd ../qserv-b
          setup -r .
       
          # and do the tests again
          cd ../qserv-run
          ./bin/qserv-start.sh
          ... do tests ...
          ./bin/qserv-stop.sh
       

      that is, as qserv-start picks up the relevant products from the environment, there's no need to rebuild/reconfigure the qserv-rundirectory each time.

        Attachments

          Issue Links

            Activity

            Hide
            salnikov Andy Salnikov added a comment -

            Hi Fabrice,

            I made few comments in the pull request. Basically I do not like what qserv-check-config does:

            • having all checks in one place does not make much sense, they should be done where they are needed
            • you are probably doing more checks than necessary, you try to avoid the cost of mis-configuration mistake (which I think is rather small) but you introduce potential code management cost instead. I believe some of those checks can be done by the services themselves, not init scripts
            Show
            salnikov Andy Salnikov added a comment - Hi Fabrice, I made few comments in the pull request. Basically I do not like what qserv-check-config does: having all checks in one place does not make much sense, they should be done where they are needed you are probably doing more checks than necessary, you try to avoid the cost of mis-configuration mistake (which I think is rather small) but you introduce potential code management cost instead. I believe some of those checks can be done by the services themselves, not init scripts
            Hide
            jammes Fabrice Jammes added a comment - - edited

            Hi Andy Salnikov,

            I tried to stick as much as possible to your recommendations. Nevertheless I kept 2 controls related to mysql-proxy, which could be removed in DM-1470.
            Moreover, I don't like to much having such duplicated checks in each init.d scripts. Would it be possible to factorize the common ones?
            If yes, would two functions be fine: one for checking qserv_run_dir, and one for both qserv_log_dir and qserv_pid_dir?

            Thanks,

            Fabrice

            Show
            jammes Fabrice Jammes added a comment - - edited Hi Andy Salnikov , I tried to stick as much as possible to your recommendations. Nevertheless I kept 2 controls related to mysql-proxy, which could be removed in DM-1470 . Moreover, I don't like to much having such duplicated checks in each init.d scripts. Would it be possible to factorize the common ones? If yes, would two functions be fine: one for checking qserv_run_dir, and one for both qserv_log_dir and qserv_pid_dir? Thanks, Fabrice
            Hide
            salnikov Andy Salnikov added a comment -

            Fabrice, I'm afraid that we are spending too much time on this simple issue, so please go ahead, implement it in a way that you think is reasonable.

            Show
            salnikov Andy Salnikov added a comment - Fabrice, I'm afraid that we are spending too much time on this simple issue, so please go ahead, implement it in a way that you think is reasonable.
            Hide
            salnikov Andy Salnikov added a comment -

            I think it should be OK one way or another.

            Show
            salnikov Andy Salnikov added a comment - I think it should be OK one way or another.
            Hide
            jammes Fabrice Jammes added a comment -

            Thanks Andy for your help in improving these scripts.

            Show
            jammes Fabrice Jammes added a comment - Thanks Andy for your help in improving these scripts.

              People

              Assignee:
              jammes Fabrice Jammes
              Reporter:
              jammes Fabrice Jammes
              Reviewers:
              Andy Salnikov
              Watchers:
              Andy Salnikov, Fabrice Jammes, Jacek Becla, Mario Juric
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  CI Builds

                  No builds found.