Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8944

migrate squash production DB to MariaDB 10.1.x

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: QA
    • Labels:
      None

      Description

      Migrate the production RDS instance from MariaDB 10.0 -> 10.1 in preparation for the next squash release.

        Attachments

          Activity

          Hide
          jhoblitt Joshua Hoblitt added a comment -

          A snapshot (in us-east-1) named pre-mariadb-10-1-update has been created as a pre-update rollback point.

          Show
          jhoblitt Joshua Hoblitt added a comment - A snapshot (in us-east-1) named pre-mariadb-10-1-update has been created as a pre-update rollback point.
          Hide
          jhoblitt Joshua Hoblitt added a comment -

          Terraform appears to have support for performing major rds engine version updates. This is currently being tested in a test env.

          Show
          jhoblitt Joshua Hoblitt added a comment - Terraform appears to have support for performing major rds engine version updates. This is currently being tested in a test env.
          Hide
          jhoblitt Joshua Hoblitt added a comment -

          The engine upgrade seems to have been successful but there are some errors in the log such as

          2017-01-09 18:11:51 2ae538081700 InnoDB: Error: Fetch of persistent statistics requested for table "mysql"."rds_heartbeat2" but the required system tables mysql.innodb_table_stats and mysql.innodb_index_stats are not present or have unexpected structure. Using transient stats instead.
          

          My guess is these are related to cloudwatch monitoring. I am going to try destroying and recreating the DB to see if these same errors are present on a freshly created instance or if there are an artifact caused by the upgrade.

          Show
          jhoblitt Joshua Hoblitt added a comment - The engine upgrade seems to have been successful but there are some errors in the log such as 2017 - 01 - 09 18 : 11 : 51 2ae538081700 InnoDB: Error: Fetch of persistent statistics requested for table "mysql" . "rds_heartbeat2" but the required system tables mysql.innodb_table_stats and mysql.innodb_index_stats are not present or have unexpected structure. Using transient stats instead. My guess is these are related to cloudwatch monitoring. I am going to try destroying and recreating the DB to see if these same errors are present on a freshly created instance or if there are an artifact caused by the upgrade.
          Hide
          jhoblitt Joshua Hoblitt added a comment -

          The errors present in an upgraded to 10.1 instance persist after a restart. I created a new 10.1 instance and these errors are not present. This appears to be a bug in the update process but it isn't clear if this is really an issue for us unless it interferes with aws's internal health monitoring.

          I am going to create a new 10.0 instance and upgrade it and then let it sit for awhile to see if rds detects any health issues.

          Show
          jhoblitt Joshua Hoblitt added a comment - The errors present in an upgraded to 10.1 instance persist after a restart. I created a new 10.1 instance and these errors are not present. This appears to be a bug in the update process but it isn't clear if this is really an issue for us unless it interferes with aws's internal health monitoring. I am going to create a new 10.0 instance and upgrade it and then let it sit for awhile to see if rds detects any health issues.
          Hide
          jhoblitt Joshua Hoblitt added a comment -

          I did another 10.0 -> 10.1 upgrade cycle and everytihng looks fine after restart of the rds instance. I'm going to proceed with making the upgrade in production.

          Show
          jhoblitt Joshua Hoblitt added a comment - I did another 10.0 -> 10.1 upgrade cycle and everytihng looks fine after restart of the rds instance. I'm going to proceed with making the upgrade in production.
          Hide
          jhoblitt Joshua Hoblitt added a comment -

          The production DB engine has been upgraded to 10.1.

          Show
          jhoblitt Joshua Hoblitt added a comment - The production DB engine has been upgraded to 10.1.

            People

            • Assignee:
              jhoblitt Joshua Hoblitt
              Reporter:
              jhoblitt Joshua Hoblitt
              Watchers:
              Adam Thornton, Angelo Fausti, Jonathan Sick, Joshua Hoblitt
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel