# Make QSERV_RUN_DIR scripts able to detect qserv install paths using eups

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
7
• Sprint:
DB_S14_10, DB_W15_11
• Team:
Data Access and Database

#### Description

Ticket related to Mario email (subject: [QSERV-L] Some points/actions from the discussion today) :

Making your "qserv data" directory independent of where qserv is installed

I think this is a big one, and largely independent of EUPS. You have a problem where you want to use one set of test data potentially with different qserv binaries (not at the same time, of course). I'd argue you should refactor the scripts generated by qserv-configure to either:

• get all their information about various paths from a single file, for example, from etc/paths.cfg.sh. Then you can easily regenerate just that file when you need to switch to a different qserv (or zookeper, or what not), or...
• refactor the generated scripts to learn from the environment which binaries to run. I.e., if $QSERV_DIR is defined, use that qserv, etc. This will let you switch binaries by simply setup-ing the new one with EUPS. The two are not mutually exclusive – e.g., all of this logic could be in etc/paths.cfg.sh, and depending on whether this is a development build or a "non-EUPS" build, it can either pick up the paths from the environment or hardcode them. Assuming you did that, your development loop may look something like this:   # assuming that qserv-configure.py has already been run  # in ../qserv-run    # do something with qserv-a clone  cd qserv-a  setup -r .  ... do some edits ...  scons    # now do the tests  cd ../qserv-run  ./bin/qserv-start.sh  ... do tests ...  ./bin/qserv-stop.sh    # now switch to qserv-b clone  cd ../qserv-b  setup -r .    # and do the tests again  cd ../qserv-run  ./bin/qserv-start.sh  ... do tests ...  ./bin/qserv-stop.sh   that is, as qserv-start picks up the relevant products from the environment, there's no need to rebuild/reconfigure the qserv-rundirectory each time. #### Attachments #### Issue Links #### Activity Hide Fabrice Jammes added a comment - At first glance, I think that making init.d scripts using eups environment isn't enough to complete this ticket. Indeed, eups products path are also used in xrootd, mysql-proxy, and client config file :  fjammes@clrlsst-dbmaster-vm:~/src/qserv/admin/templates/server/etc (u/fjammes/DM-1338)$ grep QSERV_DIR * lsp.cf:ssi.svclib {{QSERV_DIR}}/lib/libxrdsvc.so lsp.cf:ofs.osslib {{QSERV_DIR}}/lib/libxrdoss.so my-proxy.cnf:proxy-lua-script = {{QSERV_DIR}}/proxy/mysqlProxy.lua qserv-client.conf:base_dir = {{QSERV_DIR}}

Show
Fabrice Jammes added a comment - At first glance, I think that making init.d scripts using eups environment isn't enough to complete this ticket. Indeed, eups products path are also used in xrootd, mysql-proxy, and client config file : fjammes@clrlsst-dbmaster-vm:~ /src/qserv/admin/templates/server/etc (u /fjammes/DM-1338 ) \$ grep QSERV_DIR * lsp.cf:ssi.svclib {{QSERV_DIR}} /lib/libxrdsvc .so lsp.cf:ofs.osslib {{QSERV_DIR}} /lib/libxrdoss .so my-proxy.cnf:proxy-lua-script = {{QSERV_DIR}} /proxy/mysqlProxy .lua qserv-client.conf:base_dir = {{QSERV_DIR}}
Hide
Fabrice Jammes added a comment -

xrootd, mysql-proxy and qserv-czar can now use eups environment, but integration test fails. I may use git bisect to find the issue.

Show
Fabrice Jammes added a comment - xrootd, mysql-proxy and qserv-czar can now use eups environment, but integration test fails. I may use git bisect to find the issue.
Hide
Fabrice Jammes added a comment -

In init.d scripts:

• paths management can be held by eups,
• other services parameters, which are nor eups-managed or in services configuration files, are now sources from etc/sysconfig/service-name.

Fabrice

Show
Fabrice Jammes added a comment - Hi Andy Salnikov , In init.d scripts: paths management can be held by eups, other services parameters, which are nor eups-managed or in services configuration files, are now sources from etc/sysconfig/service-name. Thanks in advance for your help, Fabrice
Hide
Fabrice Jammes added a comment -

Please note that this feature may allow to switch Qserv version using for example setup qserv -t qserv, and then restart Qserv with the same QSERV_RUN_DIR and different Qserv binaries. This should work also for Qserv dependencies (xrootd, mysql-proxy, mysql, scisql).

But I think there's a few conditions for this to work (QSERV_RUN_DIR content related to a eups product has to remain compliant when product version change).
That's why I propose to test it for a few time, and then document it, when we'll have more user feedback.

Thanks,

Fabrice

Show
Fabrice Jammes added a comment - Hi Andy Salnikov , Please note that this feature may allow to switch Qserv version using for example setup qserv -t qserv , and then restart Qserv with the same QSERV_RUN_DIR and different Qserv binaries. This should work also for Qserv dependencies (xrootd, mysql-proxy, mysql, scisql). But I think there's a few conditions for this to work (QSERV_RUN_DIR content related to a eups product has to remain compliant when product version change). That's why I propose to test it for a few time, and then document it, when we'll have more user feedback. Thanks, Fabrice
Hide
Andy Salnikov added a comment -

I think we need to discuss this more before we could implement something that works. I can see several issues that are not addressed in a proposed change:

• in general init.d scripts are not supposed to use the environment (this is why recommended way to run things is via service some-service start)
• it looks like we want to satisfy requirement from both development side and production in a single solution, I'm not sure it's the best approach
• scripts in run directory may not be even compatible with the qserv release, there is no way to test for that now.

Show
Andy Salnikov added a comment - I think we need to discuss this more before we could implement something that works. I can see several issues that are not addressed in a proposed change: in general init.d scripts are not supposed to use the environment (this is why recommended way to run things is via service some-service start ) it looks like we want to satisfy requirement from both development side and production in a single solution, I'm not sure it's the best approach scripts in run directory may not be even compatible with the qserv release, there is no way to test for that now. Maybe we can talk about this again at Wednesday meeting.
Hide
Fabrice Jammes added a comment -

Hi Andy,

You're point of view is understandable and very reasonable, but I can't see how to to make point 1 and 2 compliant with previous Mario proposal (please refer to ticket description).

Point 3 is related to DM-895, and this un-covered feature isn't adressed by current ticket, but it would be good to solve this point soon.

As a reminder, the current Qserv configuration tool prototype was quickly re-written in order to enable developers to run integration tests with eups-installed Qserv. So I think it would be good, now that we have some user feedback, to define a better design, compliant with our use-cases.
Furthermore, Daniel Wang [X] has also some good ideas to improve Qserv startup scripts. So I think that your proposal to discuss this at our Wednesday meeting is very interesting.

Andy, maybe we could meet together before so that we can already propose some interesting proposals at the Wednesday meeting? Tomorrow around 10 a.m. or 2 p.m. would be fine for me.

Cheers,

Fabrice

Show
Fabrice Jammes added a comment - Hi Andy, You're point of view is understandable and very reasonable, but I can't see how to to make point 1 and 2 compliant with previous Mario proposal (please refer to ticket description). Point 3 is related to DM-895 , and this un-covered feature isn't adressed by current ticket, but it would be good to solve this point soon. As a reminder, the current Qserv configuration tool prototype was quickly re-written in order to enable developers to run integration tests with eups-installed Qserv. So I think it would be good, now that we have some user feedback, to define a better design, compliant with our use-cases. Furthermore, Daniel Wang [X] has also some good ideas to improve Qserv startup scripts. So I think that your proposal to discuss this at our Wednesday meeting is very interesting. Andy, maybe we could meet together before so that we can already propose some interesting proposals at the Wednesday meeting? Tomorrow around 10 a.m. or 2 p.m. would be fine for me. Cheers, Fabrice
Hide
Andy Salnikov added a comment -

Hi Fabrice,

I'm not sure yet I'm going to be at SLAC tomorrow, but if I will I'll let you know. Discussing that with everybody interested at Wed meeting may be even better.

Andy.

Show
Andy Salnikov added a comment - Hi Fabrice, I'm not sure yet I'm going to be at SLAC tomorrow, but if I will I'll let you know. Discussing that with everybody interested at Wed meeting may be even better. Andy.
Hide
Fabrice Jammes added a comment -

I've removed the eups environment feature, but kept sysconfig and path.cfg.sh.
What do you think of it please?

Cheers,

Show
Fabrice Jammes added a comment - Hi Andy Salnikov , I've removed the eups environment feature, but kept sysconfig and path.cfg.sh. What do you think of it please? Cheers,
Hide
Andy Salnikov added a comment -

My biggest issue now is that having multiple files in sysconfig directory does not seem to simplify things a lot. I think the idea that Mario suggested was to have all variables defined in one file (paths.cfg.sh) which is sourced by the init.scripts. If someone wants to change some paths then it's easy to find it in that file instead of searching many files for location to change. Can I propose to do this instead:

• merge all variable definitions from all files in /etc/sysconfig into one single file, it does not need to be in sysconfig, having it in /etc is OK, maybe name it differently (qserv-paths.cfg.sh?)
• that file does not need any conditional stuff or any verification that paths exist, it only defines all variables (again, the purpose is to have all variables in one place so that it is easy to change).
• all service-specific stuff and path validation should be in init.d scripts, those ideally should not have any hardcoded paths, only use whatever is defined in qserv-paths.cfg.sh

There is also one issue with lua version number hardcoded into scripts (and it was hardcoded in SConscript files before), we should find a way to determine that when we run qserv-configure (this could go into separate ticket).

Show
Andy Salnikov added a comment - Hi Fabrice, I made few comments in a pull request: https://github.com/LSST/qserv/pull/7 My biggest issue now is that having multiple files in sysconfig directory does not seem to simplify things a lot. I think the idea that Mario suggested was to have all variables defined in one file (paths.cfg.sh) which is sourced by the init.scripts. If someone wants to change some paths then it's easy to find it in that file instead of searching many files for location to change. Can I propose to do this instead: merge all variable definitions from all files in /etc/sysconfig into one single file, it does not need to be in sysconfig, having it in /etc is OK, maybe name it differently ( qserv-paths.cfg.sh ?) that file does not need any conditional stuff or any verification that paths exist, it only defines all variables (again, the purpose is to have all variables in one place so that it is easy to change). all service-specific stuff and path validation should be in init.d scripts, those ideally should not have any hardcoded paths, only use whatever is defined in qserv-paths.cfg.sh There is also one issue with lua version number hardcoded into scripts (and it was hardcoded in SConscript files before), we should find a way to determine that when we run qserv-configure (this could go into separate ticket).
Hide
Fabrice Jammes added a comment -

I tried to take in account all your remarks. Could you please review it again?

Cheers,

Show
Fabrice Jammes added a comment - Hi Andy Salnikov , I tried to take in account all your remarks. Could you please review it again? Cheers,
Hide
Andy Salnikov added a comment -

Hi Fabrice,

I made few comments in the pull request. Basically I do not like what qserv-check-config does:

• having all checks in one place does not make much sense, they should be done where they are needed
• you are probably doing more checks than necessary, you try to avoid the cost of mis-configuration mistake (which I think is rather small) but you introduce potential code management cost instead. I believe some of those checks can be done by the services themselves, not init scripts
Show
Andy Salnikov added a comment - Hi Fabrice, I made few comments in the pull request. Basically I do not like what qserv-check-config does: having all checks in one place does not make much sense, they should be done where they are needed you are probably doing more checks than necessary, you try to avoid the cost of mis-configuration mistake (which I think is rather small) but you introduce potential code management cost instead. I believe some of those checks can be done by the services themselves, not init scripts
Hide
Fabrice Jammes added a comment - - edited

I tried to stick as much as possible to your recommendations. Nevertheless I kept 2 controls related to mysql-proxy, which could be removed in DM-1470.
Moreover, I don't like to much having such duplicated checks in each init.d scripts. Would it be possible to factorize the common ones?
If yes, would two functions be fine: one for checking qserv_run_dir, and one for both qserv_log_dir and qserv_pid_dir?

Thanks,

Fabrice

Show
Fabrice Jammes added a comment - - edited Hi Andy Salnikov , I tried to stick as much as possible to your recommendations. Nevertheless I kept 2 controls related to mysql-proxy, which could be removed in DM-1470 . Moreover, I don't like to much having such duplicated checks in each init.d scripts. Would it be possible to factorize the common ones? If yes, would two functions be fine: one for checking qserv_run_dir, and one for both qserv_log_dir and qserv_pid_dir? Thanks, Fabrice
Hide
Andy Salnikov added a comment -

Fabrice, I'm afraid that we are spending too much time on this simple issue, so please go ahead, implement it in a way that you think is reasonable.

Show
Andy Salnikov added a comment - Fabrice, I'm afraid that we are spending too much time on this simple issue, so please go ahead, implement it in a way that you think is reasonable.
Hide
Andy Salnikov added a comment -

I think it should be OK one way or another.

Show
Andy Salnikov added a comment - I think it should be OK one way or another.
Hide
Fabrice Jammes added a comment -

Thanks Andy for your help in improving these scripts.

Show
Fabrice Jammes added a comment - Thanks Andy for your help in improving these scripts.

#### People

Assignee:
Fabrice Jammes
Reporter:
Fabrice Jammes
Reviewers:
Andy Salnikov
Watchers:
Andy Salnikov, Fabrice Jammes, Jacek Becla, Mario Juric