User Tools

Site Tools


maintenance:general:mariadb-10.6-upgrade

MariaDB 10.6 upgrade

This procedure is work-in-progress. Support for MariaDB 10.6 is not released yet (look for 8.0.0 builds later than 20220610)

This article describes the procedure to upgrade the installed Mariadb version to 10.6.x as there is no (semi) automatic process available. , the removal and initial install requires manual steps.

This procedure covers the MariaDB upgrade to version 10.6.x (specifically 10.6.8) from either
version 10.2.x or 10.3.x.
You can verify the current version in the System status tool or using the ck_setup.pl command.

Two install options are available:

  • semi automatic using YUM install when an internet connection is available
  • manually using RPM install when no internet connection is available.

As part of both procedures, the yum configuration will be modified to allow yum to update Mariadb installation to the latest 10.6.x release.

YUM install

This procedure can only be used when a NetYCE server is connected to the internet.

The required Linux permissions needed include root access. However, all steps can be performed as the yce functional user. Where root privileges are needed the command includes a sudo prefix.

Prepare upgrade

check db

Prepare the upgrade. Ensure the database is free from defects:

$ log_maint.pl

backup YCE db

Go the the YCE database archives tool under “Admin → System”. Create a manual archive with a description indicating it is using the older version of the MariaDB, eg “Mariadb 10.3 pre-upgrade”

backup CMDB

Do the same for the NCCM database using the Backups database archives.

stop replication

If the NetYCE server is part of a redundant master/master pair, the replication must be halted on BOTH systems. Go the the 'Admin → System → System status' tool and select a “Full report”. Scroll down to the “Database status” section and hit the “Stop replication slave” button.

Do the same for the other NetYCE database server.

yum update

It is recommended to update the Linux OS using yum before the upgrade. It allows the system to update any packages to their latest revisions. Some of them will be required for the MariaDB upgrade.

$ sudo yum update

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
epel/x86_64/metalink                                                                   |  21 kB  00:00:00
 * base: mirror.1000mbps.com
 * epel: mirrors.xtom.de
 * extras: ftp.nluug.nl
 * updates: nl.mirrors.clouvider.net
base                                                                                   | 3.6 kB  00:00:00
epel                                                                                   | 4.7 kB  00:00:00
extras                                                                                 | 2.9 kB  00:00:00
mariadb                                                                                | 3.4 kB  00:00:00
mysecureshell                                                                          |  951 B  00:00:00
updates                                                                                | 2.9 kB  00:00:00
:::
Total download size: 9.9 M
Is this ok [y/d/N]: y

Uninstall existing MariaDB

First, halt the NetYCE application, including database

$ go down

Then remove the old packages

$ rpm -qa | egrep -i 'mariadb|galera'

galera-25.3.35-1.el7.centos.x86_64
MariaDB-client-10.2.44-1.el7.centos.x86_64
MariaDB-devel-10.2.44-1.el7.centos.x86_64
MariaDB-shared-10.2.44-1.el7.centos.x86_64
MariaDB-server-10.2.44-1.el7.centos.x86_64
MariaDB-common-10.2.44-1.el7.centos.x86_64
MariaDB-compat-10.2.44-1.el7.centos.x86_64

$ --nodeps -ev <all the above packages>

Or do this all at once:
$ rpm -qa | egrep -i 'mariadb|galera' | xargs -i sudo rpm --nodeps -ev {}

yum install MariaDB 10.6

To update the Centos7 or RedHat7 system, yum is used. It uses external “repositories” to locate and update the Linux packages installed on the server. By configuring yum to use the MariaDB 10.6 repository, it will also update the MariaDB to the latest release of version 10.6.x.

Create or edit the file /etc/yum.repos.d/MariaDB.repo. It should already exist but could have a slightly different name (eg lowercase or added version). Make sure only one file with a mariadb-like name exists as yum will simply execute all files in this directory, even on conflicting versions.

The content of the file should be updated as below. On existing files only the 10.6 reference in the “baseurl” will need updating.

[MariaDB]
baseurl = http://yum.mariadb.org/10.6/centos7-amd64
enabled = 1
gpgcheck = 1
name = http://downloads.mariadb.org/mariadb/repositories/
gpgkey = https://yum.mariadb.org/RPM-GPG-KEY-MariaDB

With this yum configuration file in place, the MariaDB 10.6 installation can also be executed using yum:

$ sudo yum install mariadb-server

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.1000mbps.com
 * epel: mirrors.xtom.de
 * extras: ftp.nluug.nl
 * updates: nl.mirrors.clouvider.net
Package mariadb-server is obsoleted by MariaDB-server, trying to install MariaDB-server-10.6.8-1.el7.centos.x86_64 instead
Resolving Dependencies
--> Running transaction check
---> Package MariaDB-server.x86_64 0:10.6.8-1.el7.centos will be installed
--> Processing Dependency: MariaDB-common >= 10.6.1 for package: MariaDB-server-10.6.8-1.el7.centos.x86_64
:::
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================
 Package                     Arch                Version                           Repository            Size
==============================================================================================================
Installing:
 MariaDB-server              x86_64              10.6.8-1.el7.centos               mariadb               25 M
Installing for dependencies:
 MariaDB-client              x86_64              10.6.8-1.el7.centos               mariadb               14 M
 MariaDB-common              x86_64              10.6.8-1.el7.centos               mariadb               81 k
 MariaDB-compat              x86_64              10.6.8-1.el7.centos               mariadb              2.2 M
 galera-4                    x86_64              26.4.11-1.el7.centos              mariadb              9.9 M
 libpmem                     x86_64              1.5.1-2.1.el7                     base                  59 k
 pcre2                       x86_64              10.23-2.el7                       base                 201 k
 socat                       x86_64              1.7.3.2-2.el7                     base                 290 k

Transaction Summary
==============================================================================================================
Install  1 Package (+7 Dependent packages)

Total download size: 52 M
Installed size: 222 M
Is this ok [y/d/N]: y
:::
Installed:
  MariaDB-server.x86_64 0:10.6.8-1.el7.centos

Dependency Installed:
  MariaDB-client.x86_64 0:10.6.8-1.el7.centos           MariaDB-common.x86_64 0:10.6.8-1.el7.centos
  MariaDB-compat.x86_64 0:10.6.8-1.el7.centos           galera-4.x86_64 0:26.4.11-1.el7.centos
  libpmem.x86_64 0:1.5.1-2.1.el7                        pcre2.x86_64 0:10.23-2.el7
  socat.x86_64 0:1.7.3.2-2.el7

Complete!

The new MariaDB is now installed but will fail to run as its default configuration file will need to be replaced.

Continue the installation using the section MariaDB configuration section below.

RPM install

Us this procedure when the NetYCE server cannot connect to the internet. The required packages need to be downloaded and installed manually.

The required Linux permissions needed include root access. However, all steps can be performed as the yce functional user. Where root privileges are needed the command includes a sudo prefix.

Downloads

MariaDB 10.6.x

Retrieve latest release of MariaDB 10.6 for CentOS7/RedHat7 on x86_64

point browser to: https://mirror.mariadb.org/yum/10.6/centos7-amd64/rpms/

find the latests versions (now 10.6.8) of the rpms for:

  • server → MariaDB-server-10.6.8-1.el7.centos.x86_64.rpm
  • client → MariaDB-client-10.6.8-1.el7.centos.x86_64.rpm
  • common → MariaDB-common-10.6.8-1.el7.centos.x86_64.rpm
  • compat → MariaDB-compat-10.6.8-1.el7.centos.x86_64.rpm
  • shared → MariaDB-shared-10.6.8-1.el7.centos.x86_64.rpm
  • galera → galera-4-26.4.11-1.el7.centos.x86_64.rpm

Note that the 'devel' package is optional for customer systems. It is only required for NetYCE development servers

  • devel → MariaDB-devel-10.6.8-1.el7.centos.x86_64.rpm

Using these filenames, retrieve them and download or copy them to the NetYCE server:

 
$ mkdir /var/tmp/mariadb-10.6.8
$ cd /var/tmp/mariadb-10.6.8

$ wget https://mirror.mariadb.org/yum/10.6/centos7-amd64/rpms/MariaDB-server-10.6.8-1.el7.centos.x86_64.rpm
$ wget https://mirror.mariadb.org/yum/10.6/centos7-amd64/rpms/MariaDB-client-10.6.8-1.el7.centos.x86_64.rpm
$ wget https://mirror.mariadb.org/yum/10.6/centos7-amd64/rpms/galera-4-26.4.11-1.el7.centos.x86_64.rpm
.. etc

dependencies

The MariaDb 10.6 installation depends on some other packages that need to be installed at the same time, or installed earlier.

point browser to: http://mirror.centos.org/centos-7/7/os/x86_64/Packages/

find the latests X68_64 versions of the rpms for:

  • libpmem → libpmem-1.5.1-2.1.el7.x86_64.rpm
  • pcre2 → pcre2-10.23-2.el7.x86_64.rpm
  • socat → socat-1.7.3.2-2.el7.x86_64.rpm
$ cd /var/tmp/mariadb-10.6.8

$ wget http://mirror.centos.org/centos-7/7/os/x86_64/Packages/libpmem-1.5.1-2.1.el7.x86_64.rpm
$ wget http://mirror.centos.org/centos-7/7/os/x86_64/Packages/pcre2-10.23-2.el7.x86_64.rpm
$ wget http://mirror.centos.org/centos-7/7/os/x86_64/Packages/socat-1.7.3.2-2.el7.x86_64.rpm

Additionally, glibc must be installed and must be of a specific version. The current (for MariaDB 10.6.8) glibc version is 2.17-326.el7_9 for x86_64. To check which glibc version is installed:

$ rpm -qa | grep -i glibc

glibc-headers-2.17-326.el7_9.x86_64
glibc-2.17-326.el7_9.x86_64
glibc-common-2.17-326.el7_9.x86_64
glibc-devel-2.17-326.el7_9.x86_64

If your current version is lower than the 2.17.326, download these packages from: http://centos.mirror.transip.nl/7/updates/x86_64/Packages

  • glibc → glibc-2.17-326.el7_9.x86_64.rpm
  • glibc-devel → glibc-devel-2.17-326.el7_9.x86_64.rpm
  • glibc-common → glibc-common-2.17-326.el7_9.x86_64.rpm
  • glibc-headers → glibc-headers-2.17-326.el7_9.x86_64.rpm
$ cd /var/tmp/mariadb-10.6.8

$ wget http://centos.mirror.transip.nl/7/updates/x86_64/Packages/glibc-2.17-326.el7_9.x86_64.rpm
$ wget http://centos.mirror.transip.nl/7/updates/x86_64/Packages/glibc-common-2.17-326.el7_9.x86_64.rpm
$ wget http://centos.mirror.transip.nl/7/updates/x86_64/Packages/glibc-devel-2.17-326.el7_9.x86_64.rpm
$ wget http://centos.mirror.transip.nl/7/updates/x86_64/Packages/glibc-headers-2.17-326.el7_9.x86_64.rpm

Prepare upgrade

check db

Prepare the upgrade. Ensure the database is free from defects:

$ log_maint.pl

backup YCE db

Go the the YCE database archives tool under “Admin → System”. Create a manual archive with a description indicating it is using the older version of the MariaDB, eg “Mariadb 10.3 pre-upgrade”

backup CMDB

Do the same for the NCCM database using the Backups database archives.

stop replication

If the NetYCE server is part of a redundant master/master pair, the replication must be halted on BOTH systems. Go the the 'Admin → System → System status' tool and select a “Full report”. Scroll down to the “Database status” section and hit the “Stop replication slave” button.

Do the same for the other NetYCE database server.

Uninstall existing MariaDB

First, halt the NetYCE application, including database

$ go down

Then remove the old packages

$ rpm -qa | egrep -i 'mariadb|galera'

galera-25.3.35-1.el7.centos.x86_64
MariaDB-client-10.2.44-1.el7.centos.x86_64
MariaDB-devel-10.2.44-1.el7.centos.x86_64
MariaDB-shared-10.2.44-1.el7.centos.x86_64
MariaDB-server-10.2.44-1.el7.centos.x86_64
MariaDB-common-10.2.44-1.el7.centos.x86_64
MariaDB-compat-10.2.44-1.el7.centos.x86_64

$ --nodeps -ev <all the above packages>

Or do this all at once:
$ rpm -qa | egrep -i 'mariadb|galera' | xargs -i sudo rpm --nodeps -ev {}

YUM configuration

To update the Centos7 or RedHat7 system, yum is used. It uses external “repositories” to locate and update the Linux packages installed on the server. By configuring yum to use the MariaDB 10.6 repository, it will also update the MariaDB to the latest release of version 10.6.x.

Create or edit the file /etc/yum.repos.d/MariaDB.repo. It should already exist but could have a slightly different name (eg lowercase or added version). Make sure only one file with a mariadb-like name exists as yum will simply execute all files in this directory, even on conflicting versions.

The content of the file should be updated as below. On existing files only the 10.6 reference in the “baseurl” will need updating.

[MariaDB]
baseurl = http://yum.mariadb.org/10.6/centos7-amd64
enabled = 1
gpgcheck = 1
name = http://downloads.mariadb.org/mariadb/repositories/
gpgkey = https://yum.mariadb.org/RPM-GPG-KEY-MariaDB

rpm install MariaDB 10.6

From the location of the downloaded rpm files

$ cd /var/tmp/mariadb-10.6.8

Install all packages at once. Trying to do these one-by-one will result in circular dependencies.

$ sudo rpm -Uvh *.rpm

Preparing...                          ################################# [100%]
Updating / installing...
   1:MariaDB-compat-10.6.8-1.el7.cento################################# [ 10%]
   2:MariaDB-common-10.6.8-1.el7.cento################################# [ 20%]
   3:libpmem-1.5.1-2.1.el7            ################################# [ 30%]
   4:pcre2-10.23-2.el7                ################################# [ 40%]
   5:MariaDB-client-10.6.8-1.el7.cento################################# [ 50%]
   6:MariaDB-shared-10.6.8-1.el7.cento################################# [ 60%]
   7:socat-1.7.3.2-2.el7              ################################# [ 70%]
   8:galera-4-26.4.9-1.el7.centos     ################################# [ 80%]
   9:MariaDB-server-10.6.8-1.el7.cento################################# [ 90%]
  10:MariaDB-devel-10.6.8-1.el7.centos################################# [100%]

The new MariaDB is now installed but will fail to run as its default configuration file will need to be replaced.

Continue the installation using the section MariaDB configuration section below.

MariaDB configuration

First the NetYCE process monitor needs to be started:

$ go start yce_psmon
-- starting Daemon 'yce_psmon'
yce_psmon:
  start: /usr/bin/sudo /usr/bin/systemctl start yce_psmon.service
  wait start 'yce_psmon': 3816
done

Then, to re-create the configuration file for the MariaDB, run the yce_setup command with the -r (regenerate) option:

$ yce_setup.pl -r

This command will regenerate the configuration files for all NetYCE components and will (re)start the relevant processes. It will also start the MaraDB server.

During this initial run of yce_setup.pl some warnings will be issued due to the missing mysql/mariadb.

To complete the configuration, re-run the yce_setup.pl -r command. As the database is now reachable, the warnings should disappear.

$ yce_setup.pl - r

Verifications

Verify the NetYCE setup acknowledges the MariaDB dependencies are met:

$ ck_setup.pl

-- OK. Have a 'x86_64' architecture
-- OK. Have a supported EL7 distribution: CentOS EL7 7.9.2009
-- OK. Have YCEperl version '8.0.0'
-- OK. Found current '5.32.0' perl CORE/libperl.so
-- OK. Found link to previous '5.24.0' perl CORE/libperl.so
-- OK. Mariadb version '10.6' is supported for EL7
-- OK. Found required mysql library 'libmariadbd.so.19'

Next, the upgrade will require some updates to the database tables. This command will check and optimize all tables after the initial 'mysql_upgrade' that is part of the 10.6 distribution is completed:

$ mysql_repair.sh

When done, these commands are part of the daily maintenance and should produce no errors:

$ log_maint.pl

$ logs_daily.pl

$ nccm_daily.pl

To check if the database can be properly accessed, use the ck_license command:

$ ck_license -X
This will output license details like:

OK: Version '8.0.0' matches license for '8'
OK: License will not expire until '20221231'
OK: Architecture license Full. Found '1' servers
OK: Users license 'Unlim' sufficient for '0' users
OK: Scheduled jobs license 'Unlim' sufficient for '0' jobs
OK: Client-types license 'Unlim' sufficient for '5' client-types
OK: Service-types license 'Unlim' sufficient for '11' service-types
OK: Node license 'Unlim' sufficient for '414' nodes
OK: Port license 'Unlim' sufficient for '747' ports
OK: Service license 'Unlim' sufficient for '43' services
OK: Node-compliance license 'Unlim' sufficient for '0' nodes
OK: Mpls_vrf license 'Unlim' sufficient for '177' mpls_vrfs

Restoring replication

After - and only after - both NetYCE servers are upgraded to use MariaDB 10.6 can the master/master replication between the servers be restored.

This involves the (re)configuration of the replication between the servers using the information provided in an earlier yce_setup.pl session.

But as both systems will have been unable to communicate for a while, the databases are almost certainly out-of-sync. To restore synchronization, use the 'ck_dbsync'' tool to verify and synchronize the databases. Do not allow users to login or API requests while this synchronization is in progress. Depending on database size, this can take half an hour to several hours to complete.

$ ck_dbsync.pl -R -x

When done, the synchronization status should look like this:

$ ck_dbsync.pl
2022-06-08 17:49:23 === Started ===
2022-06-08 17:49:23 Defaulting to replication status (try -h for help)
2022-06-08 17:49:23 Using local PRI and remote SEC setup:
2022-06-08 17:49:23   PRImary:   172.17.0.24
2022-06-08 17:49:23   SECondary: 172.17.0.25
2022-06-08 17:49:23 Connected to both databases
2022-06-08 17:49:23 1/1 Sync status
2022-06-08 17:49:23 1/1 Replication status
2022-06-08 17:49:23 1/1   PRI slave IO : Yes
2022-06-08 17:49:23 1/1   PRI slave SQL: Yes
2022-06-08 17:49:23 1/1   SEC slave IO : Yes
2022-06-08 17:49:23 1/1   SEC slave SQL: Yes
2022-06-08 17:49:23 === Completed ===

Create 10.6 archives

To complete the upgrade save the upgraded 10.6 databases using the archiving tools.

Go the the YCE database archives tool under “Admin → System”. Create a manual archive with a description indicating it is using the new version of the MariaDB, eg “Mariadb 10.6 post-upgrade”

Do the same for the Backups archive.

LDAP: couldn't connect to LDAP server
maintenance/general/mariadb-10.6-upgrade.txt · Last modified: 2022/06/14 10:06 by yspeerte