Migrating databases from a third-party Redis cluster
For data migration, Redis uses a logical dump: this is a file with a sequence of commands to restore the state of databases in the cluster. There are several ways to create a dump. The following example will use redis-dump-go
Note
A binary RDB dump can't be used for migration, because Managed Service for Redis doesn't permit accessing file systems on cluster hosts.
To migrate Redis databases from the source cluster to the target cluster:
-
(Optional) Upload the dump to the intermediate virtual machine.
You must transfer data to Compute Cloud using a virtual machine if:
- Your Managed Service for Redis cluster is not accessible from the internet.
- Your hardware or connection to the cluster in Nebius AI is not very reliable.
If you no longer need the resources you created, delete them.
Getting started
Prepare the infrastructure
-
Create a Managed Service for Redis cluster with any suitable configuration. To connect to a cluster from a user's local machine rather than a Nebius AI cloud network, enable TLS support and public host access when creating your cluster.
-
(Optional) Create an intermediate Linux virtual machine in Compute Cloud on the same network as the Managed Service for Redis cluster using the following configuration:
-
Under Image/boot disk selection, select Operating systems → Ubuntu 20.04.
-
Under Network settings:
- Public IP: Select the checkbox.
-
Install additional software
-
(Optional) Install utilities on the local machine for downloading and uploading files over SSH, such as:
-
Make sure that GNU Screen
is installed on the source cluster.It might take a long time to create and restore a dump. To keep these processes alive when your SSH session times out, start them using this utility. If your SSH connection breaks while creating or restoring the dump, reconnect and restore the session state using the command:
screen -R
Connect to the source cluster and create a logical dump
-
Connect to the source cluster's master host via SSH.
-
Download the archive with the
redis-dump-go
utility from the project page . In the following examples, we'll use version0.5.1
.wget https://github.com/yannh/redis-dump-go/releases/download/v0.5.1/redis-dump-go_0.5.1_linux_amd64.tar.gz
-
Unpack the archive to the current directory:
tar xf redis-dump-go_0.5.1_linux_amd64.tar.gz
-
Get familiar with the utility launch parameters:
./redis-dump-go -h
-
If connecting to the Redis cluster requires a password, enter it in the
REDISDUMPGO_AUTH
environment variable:export REDISDUMPGO_AUTH="<Redis password>"
-
Start an interactive
screen
session:screen
-
Launch the creation of a logical dump:
./redis-dump-go \ -host <master host IP or FQDN in a Redis cluster> \ -port <port Redis> > <dump file>
Tip
As the dump is created, the number of processed keys is shown on the screen. Remember or write down the last output value: you'll need it to check that the dump has been recovered completely in the target cluster.
-
When the dump has been created, download it to your computer.
-
End the interactive
screen
session:exit
(Optional) Upload the dump to the intermediate virtual machine
-
Upload the dump from your computer to an intermediate virtual machine however is convenient.
Restore the dump on the target cluster
-
Connect to the cluster and run an interactive
screen
session:screen
-
Start the dump recovery process:
Connecting without using TLSConnecting via TLSBefore connecting, install the dependencies:
sudo apt update && sudo apt install -y redis-tools
Connecting via Sentinel
host=$(redis-cli \ -h <FQDN of any host Redis> \ -p 26379 \ sentinel \ get-master-addr-by-name \ no-shards-no-tls | head -n 1) redis-cli \ -h ${host} \ -p 6379 \ -a <target cluster password> \ --pipe < <dump file>
Connecting directly to the master host
redis-cli \ -h <FQDN of any host> \ -p 6379 \ -a <target cluster password> \ --pipe < <dump file>
Instead of the master host's FQDN, you can use special FQDNs.
Before connecting, install the dependencies:
Build the
redis-tools
utility with TLS support in one of two ways:-
From a repository
-
Connect a repository:
sudo apt-add-repository ppa:redislabs/redis
Packages in this repository have already been built with the
BUILD_TLS=yes
flag. -
Install the utility:
sudo apt update && sudo apt install -y redis-tools
-
-
Manually
Go to the directory you want to download the distribution to. Download the stable version of the utility, then build and install it:
wget https://download.redis.io/redis-stable.tar.gz && \ tar -xzvf redis-stable.tar.gz && \ cd redis-stable && \ make BUILD_TLS=yes && \ sudo make install && \ sudo cp ./src/redis-cli /usr/bin/
Connecting via Sentinel
host=$(redis-cli \ -h <FQDN of any Redis host> \ -p 26379 \ sentinel \ get-master-addr-by-name \ no-shards-tls | head -n 1) redis-cli \ -h ${host} \ -p 6380 \ -a <target cluster password> \ --tls \ --cacert ~/.redis/NebiusAICA.crt \ --pipe < <dump file>
Connecting directly to the master host
redis-cli \ -h c-<cluster ID>.rw.mdb.nemax.nebius.cloud \ -p 6380 \ -a <target cluster password> \ --tls \ --cacert ~/.redis/NebiusAICA.crt \ --pipe < <dump file>
Instead of the master host's FQDN, you can use special FQDNs.
-
-
End the interactive
screen
session:exit
Make sure that the dump is restored completely
- In the management console
, select the folder containing the cluster to restore. - In the list of services, select Managed Service for Redis.
- Click the name of the desired cluster and open the Monitoring tab.
Pay attention to the DB Keys chart showing the number of keys stored in the cluster.
The total number of keys in the cluster must be equal to the number of keys processed by the redis-dump-go
utility when creating the dump.
Delete the resources you created
Delete the resources you no longer need to avoid being charged for them:
- Delete the Managed Service for Redis cluster.
- If you created an intermediate virtual machine, delete it.