Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4383

Avoid restarting czar when empty chunk list changes

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Qserv
    • Labels:

      Description

      Currently czar caches empty chunk list after it reads the list from file. This complicates things when we need to update the list, integration test for example has to restart czar process after it loads new data to make sure that czar updates its cached list. Would be nice to have simpler mechanism to resetting cached list in czar without restarting it completely. It could be done via special query (abusing FLUSH for example) or via sending signal (problematic if czar runs remotely).

      This can be potentially useful even after we replace empty chunk list file with some other mechanism as I expect that cache will stay around even for that.

        Attachments

          Issue Links

            Activity

            Hide
            salnikov Andy Salnikov added a comment - - edited

            I can think of a few different ways to solve this:
            1. send special query to czar, something like FLUSH QSERV_EMPTY_CHUNKS [FOR database], this needs client to know about mysql connection properties for proxy
            2. use wmgr to do the same, there will be new special method to call and wmgr will then need to talk to czar again and ask it to flush its internal cache
            3. use both methods

            Data loader currently does not need any mysql communication with czar and it does not have mysql connection parameters/credentials, but it talks to wmgr, so for data loader option 2 is preferred. To implement wmgr-to-czar communication the easiest is likely option 1, as we do not have any other side channel to talk to czar.

            Show
            salnikov Andy Salnikov added a comment - - edited I can think of a few different ways to solve this: 1. send special query to czar, something like FLUSH QSERV_EMPTY_CHUNKS [FOR database] , this needs client to know about mysql connection properties for proxy 2. use wmgr to do the same, there will be new special method to call and wmgr will then need to talk to czar again and ask it to flush its internal cache 3. use both methods Data loader currently does not need any mysql communication with czar and it does not have mysql connection parameters/credentials, but it talks to wmgr, so for data loader option 2 is preferred. To implement wmgr-to-czar communication the easiest is likely option 1, as we do not have any other side channel to talk to czar.
            Hide
            salnikov Andy Salnikov added a comment -

            Thinking about this again, I believe we should be talking not about "empty chunks" but "chunks" in general. Czar wants to know if particular chunk is empty/non-empty before scheduling it to optimize things. "Empty chunks list" is current implementation of that process, but implementation could change (I think it may be done faster with a full bitmap rather than set of chunk IDs). I want to change FLUSH QSERV_EMPTY_CHUNKS to FLUSH QSERV_CHUNKS_CACHE to avoid mentioning "empty" and say that we are flushing cache only, not a full chunk map.

            Show
            salnikov Andy Salnikov added a comment - Thinking about this again, I believe we should be talking not about "empty chunks" but "chunks" in general. Czar wants to know if particular chunk is empty/non-empty before scheduling it to optimize things. "Empty chunks list" is current implementation of that process, but implementation could change (I think it may be done faster with a full bitmap rather than set of chunk IDs). I want to change FLUSH QSERV_EMPTY_CHUNKS to FLUSH QSERV_CHUNKS_CACHE to avoid mentioning "empty" and say that we are flushing cache only, not a full chunk map.
            Hide
            salnikov Andy Salnikov added a comment -

            Fritz, can you review this if you have a minute. It's not too terrible I believe.

            Show
            salnikov Andy Salnikov added a comment - Fritz, can you review this if you have a minute. It's not too terrible I believe.
            Hide
            fritzm Fritz Mueller added a comment -

            Looks good to me. One Q in PR – what would you think about (ab)using stored procedure CALL syntax for this as an alternative to extending the FLUSH grammar – better/worse?

            Show
            fritzm Fritz Mueller added a comment - Looks good to me. One Q in PR – what would you think about (ab)using stored procedure CALL syntax for this as an alternative to extending the FLUSH grammar – better/worse?
            Hide
            salnikov Andy Salnikov added a comment -

            Thanks for review! As I said on PR I do not really know which one is better/worse now. Let's keep it as it is now but we may need to think/talk more about how we are going to extend SQL. Once we know more we can always change this particular query to something else, there is no problem with compatibility as this query is qserv-internal and not for end-users.

            Merged and pushed.

            Show
            salnikov Andy Salnikov added a comment - Thanks for review! As I said on PR I do not really know which one is better/worse now. Let's keep it as it is now but we may need to think/talk more about how we are going to extend SQL. Once we know more we can always change this particular query to something else, there is no problem with compatibility as this query is qserv-internal and not for end-users. Merged and pushed.

              People

              • Assignee:
                salnikov Andy Salnikov
                Reporter:
                salnikov Andy Salnikov
                Reviewers:
                Fritz Mueller
                Watchers:
                Andy Salnikov, Fritz Mueller
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel