Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-30080

Investigate MirrorMaker 2 behavior under partial failure of Kafka Connect tasks

    XMLWordPrintable

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      3 of 10 Mirror Maker tasks went down after Kafka Connect restarted on May 5. This caused MirrorMaker 2 to lose data as shown in the attached screenshots.

      I would expect Kafka Connect to redistribute the work to the running MIrror Maker 2 tasks instead.

      [2021-05-05 20:11:23,226] ERROR WorkerSourceTask{id=mirror-source-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)
      [2021-05-05 20:11:23,225] ERROR WorkerSourceTask{id=mirror-source-1} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)
      [2021-05-05 20:11:23,225] ERROR WorkerSourceTask{id=mirror-source-2} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)
      

        Attachments

          Issue Links

            Activity

            Hide
            afausti Angelo Fausti added a comment -

            Have to verify this again when MM2 is running in Sasquatch. Moving to my backlog epic for now.

            Show
            afausti Angelo Fausti added a comment - Have to verify this again when MM2 is running in Sasquatch. Moving to my backlog epic for now.

              People

              Assignee:
              afausti Angelo Fausti
              Reporter:
              afausti Angelo Fausti
              Watchers:
              Angelo Fausti
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:

                  Jenkins

                  No builds found.