深入淺析Redis 集群伸縮原理

更新時(shí)間：2021年05月15日 09:51:31 作者：LB

Redis 集群提供了靈活的節(jié)點(diǎn)擴(kuò)容和收縮方案。在不影響集群對(duì)外服務(wù)的情況下，可以為集群添加節(jié)點(diǎn)進(jìn)行擴(kuò)容，也可以下線部分節(jié)點(diǎn)進(jìn)行縮容，接下來(lái)通過(guò)本文給大家分享Redis 集群伸縮原理，感興趣的朋友一起看看吧

一、集群擴(kuò)容

1. 手動(dòng)擴(kuò)容

(1) 準(zhǔn)備節(jié)點(diǎn) 9007，并加入集群

192.168.11.40:9001> cluster meet 192.168.11.40 9007

【注意】若 cluster meet 加入已存在于其它集群的節(jié)點(diǎn)，會(huì)導(dǎo)致集群合并，造成數(shù)據(jù)錯(cuò)亂！。建議使用 redis-cli 的 add-node：

# 若節(jié)點(diǎn)已加入其它集群或包含數(shù)據(jù)，會(huì)報(bào)錯(cuò)
add-node    new_host:new_port existing_host:existing_port
            --cluster-slave  # 直接添加為從節(jié)點(diǎn)
            --cluster-master-id <arg>  # 從節(jié)點(diǎn)對(duì)應(yīng)的主節(jié)點(diǎn)id

(2) 遷移槽和數(shù)據(jù)

槽在遷移過(guò)程中集群可以正常提供讀寫(xiě)服務(wù)

首先確定原有節(jié)點(diǎn)的哪些槽需要遷移到新節(jié)點(diǎn)。確保每個(gè)節(jié)點(diǎn)負(fù)責(zé)相似數(shù)量的槽，保證各節(jié)點(diǎn)的數(shù)據(jù)均勻

槽是 Redis 集群管理數(shù)據(jù)的基本單位。數(shù)據(jù)遷移是逐槽進(jìn)行的

槽遷移流程：

目標(biāo)節(jié)點(diǎn)準(zhǔn)備導(dǎo)入槽的數(shù)據(jù)：目標(biāo)節(jié)點(diǎn)執(zhí)行cluster setslot {slot} importing {sourceNodeId}
源節(jié)點(diǎn)準(zhǔn)備遷出槽的數(shù)據(jù)：源節(jié)點(diǎn)執(zhí)行cluster setslot {slot} migrating {targetNodeId}
獲取 count 個(gè)屬于槽 slot 的鍵：源節(jié)點(diǎn)執(zhí)行cluster getkeysinslot {slot} {count}
遷移鍵：源節(jié)點(diǎn)執(zhí)行migrate {targetIp} {targetPort} "" 0 {timeout} keys {keys...}，把鍵通過(guò)流水線（pipeline）機(jī)制批量遷移到目標(biāo)節(jié)點(diǎn)。Redis3.0.6 后才支持批量遷移
重復(fù)上兩步，直到槽下所有的鍵值數(shù)據(jù)遷移到目標(biāo)節(jié)點(diǎn)
向集群所有主節(jié)點(diǎn)通知槽被分配給目標(biāo)節(jié)點(diǎn)：集群內(nèi)所有主節(jié)點(diǎn)執(zhí)行cluster setslot {slot} node {targetNodeId}

內(nèi)部偽代碼：

def move_slot(source,target,slot):
    # 目標(biāo)節(jié)點(diǎn)準(zhǔn)備導(dǎo)入槽
    target.cluster("setslot",slot,"importing",source.nodeId);
    # 目標(biāo)節(jié)點(diǎn)準(zhǔn)備全出槽
    source.cluster("setslot",slot,"migrating",target.nodeId);
    while true :
        # 批量從源節(jié)點(diǎn)獲取鍵
        keys = source.cluster("getkeysinslot",slot,pipeline_size);
        if keys.length == 0:
            # 鍵列表為空時(shí)，退出循環(huán)
            break;
        # 批量遷移鍵到目標(biāo)節(jié)點(diǎn)
        source.call("migrate",target.host,target.port,"",0,timeout,"keys",keys);
        # 向集群所有主節(jié)點(diǎn)通知槽被分配給目標(biāo)節(jié)點(diǎn)
        for node in nodes:
            if node.flag == "slave":
                continue;
            node.cluster("setslot",slot,"node",target.nodeId);

(3) 將 9001 的槽 4096 遷移到 9007 中

準(zhǔn)備數(shù)據(jù)

192.168.11.40:9001> set key:test:5028 value:5028
192.168.11.40:9001> set key:test:68253 value:68253

目標(biāo)節(jié)點(diǎn)準(zhǔn)備工作

192.168.11.40:9007> cluster nodes
8ccdb0963411ebd05ce21952bdd4b7597825afdc 192.168.11.40:9001@19001 master - 0 1620928869000 2 connected 0-5461
bb1bb0f5f9e0ee67846ba8ec94a38da700e2e80d 192.168.11.40:9007@19007 myself,master - 0 1620928868000 0 connected
...
# 9007 準(zhǔn)備導(dǎo)入槽 4096 的數(shù)據(jù)
192.168.11.40:9007> cluster setslot 4096 importing 8ccdb0963411ebd05ce21952bdd4b7597825afdc
OK
# 槽 4096 已開(kāi)啟導(dǎo)入狀態(tài)
192.168.11.40:9007> cluster nodes
bb1bb0f5f9e0ee67846ba8ec94a38da700e2e80d 192.168.11.40:9007@19007 myself,master - 0 1620928959000 0 connected [4096-<-8ccdb0963411ebd05ce21952bdd4b7597825afdc]
...

源節(jié)點(diǎn)準(zhǔn)備工作

# 9001 準(zhǔn)備導(dǎo)出槽 4096 數(shù)據(jù)
192.168.11.40:9001> cluster setslot 4096 migrating bb1bb0f5f9e0ee67846ba8ec94a38da700e2e80d
OK
# 槽 4096 已開(kāi)啟導(dǎo)出狀態(tài)
192.168.11.40:9001> cluster nodes
8ccdb0963411ebd05ce21952bdd4b7597825afdc 192.168.11.40:9001@19001 myself,master - 0 1620929179000 2 connected 0-5461 [4096->-bb1bb0f5f9e0ee67846ba8ec94a38da700e2e80d]
...

導(dǎo)出數(shù)據(jù)

# 獲取 100 個(gè)屬于槽 4096 的鍵
192.168.11.40:9001> cluster getkeysinslot 4096 100
1) "key:test:5028"
2) "key:test:68253"
# 查看數(shù)據(jù)
192.168.11.40:9001> mget key:test:5028 key:test:68253
1) "value:5028"
2) "value:68253"
# 遷移這2個(gè)鍵：migrate 命令保證了每個(gè)鍵遷移過(guò)程的原子性
192.168.11.40:9001> migrate 192.168.11.40 9007 "" 0 5000 keys key:test:5028 key:test:68253
OK
# 再次查詢(xún)會(huì)報(bào) ASK 錯(cuò)誤：引導(dǎo)客戶(hù)端找到數(shù)據(jù)所在的節(jié)點(diǎn)
192.168.11.40:9001> mget key:test:5028 key:test:68253
(error) ASK 4096 192.168.11.40:9007

通知所有主節(jié)點(diǎn)：槽 4096 指派給 9007

192.168.11.40:9001> cluster setslot 4096 node bb1bb0f5f9e0ee67846ba8ec94a38da700e2e80d
192.168.11.40:9002> cluster setslot 4096 node bb1bb0f5f9e0ee67846ba8ec94a38da700e2e80d
192.168.11.40:9003> cluster setslot 4096 node bb1bb0f5f9e0ee67846ba8ec94a38da700e2e80d
192.168.11.40:9007> cluster setslot 4096 node bb1bb0f5f9e0ee67846ba8ec94a38da700e2e80d

查看最終結(jié)果

192.168.11.40:9007> cluster nodes
8ccdb0963411ebd05ce21952bdd4b7597825afdc 192.168.11.40:9001@19001 master - 0 1620931743303 7 connected 0-4095 4097-5461
bb1bb0f5f9e0ee67846ba8ec94a38da700e2e80d 192.168.11.40:9007@19007 myself,master - 0 1620931741000 8 connected 4096
...

2. 使用 redis-cli 擴(kuò)容

redis-cli 提供了槽重分片功能

reshard 命令參數(shù)詳解：

reshard    host:port  # 集群內(nèi)任意節(jié)點(diǎn)地址
           --cluster-from <arg>  # 源節(jié)點(diǎn)id，逗號(hào)分隔
           --cluster-to <arg>  # 目標(biāo)節(jié)點(diǎn)id，只有一個(gè)
           --cluster-slots <arg>  # 遷移多少個(gè)槽
           --cluster-yes  # 確認(rèn)執(zhí)行reshard
           --cluster-timeout <arg>  # 每次 migrate 操作的超時(shí)時(shí)間，默認(rèn) 60000ms
           --cluster-pipeline <arg>  # 每次批量遷移鍵的數(shù)量，默認(rèn) 10
           --cluster-replace

將 9001、9002、9003 的槽遷移到 9007，共遷移 4096 個(gè)

$ /usr/local/redis/bin/redis-cli --cluster reshard 192.168.11.40:9001
M: 8ccdb0963411ebd05ce21952bdd4b7597825afdc 192.168.11.40:9001
   slots:[0-4095],[4097-5461] (5461 slots) master
   1 additional replica(s)
M: bb1bb0f5f9e0ee67846ba8ec94a38da700e2e80d 192.168.11.40:9007
   slots:[4096] (1 slots) master
...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 4096
What is the receiving node ID? bb1bb0f5f9e0ee67846ba8ec94a38da700e2e80d
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1: 8ccdb0963411ebd05ce21952bdd4b7597825afdc
Source node #2: 5786e3237c7fa413ed22465d15be721f95e72cfa
Source node #3: 85ceb9826e8aa003169c46fb4ba115c72002d4f9
Source node #4: done
    Moving slot 0 from 8ccdb0963411ebd05ce21952bdd4b7597825afdc
    ...
    Moving slot 12287 from 85ceb9826e8aa003169c46fb4ba115c72002d4f9
Do you want to proceed with the proposed reshard plan (yes/no)? yes
Moving slot 0 from 192.168.11.40:9001 to 192.168.11.40:9007:
...
Moving slot 12287 from 192.168.11.40:9003 to 192.168.11.40:9007:

查看最終結(jié)果

192.168.11.40:9007> cluster nodes
8ccdb0963411ebd05ce21952bdd4b7597825afdc 192.168.11.40:9001@19001 master - 0 1620933907753 7 connected 1366-4095 4097-5461
5786e3237c7fa413ed22465d15be721f95e72cfa 192.168.11.40:9002@19002 master - 0 1620933906733 1 connected 6827-10922
85ceb9826e8aa003169c46fb4ba115c72002d4f9 192.168.11.40:9003@19003 master - 0 1620933905000 3 connected 12288-16383
bb1bb0f5f9e0ee67846ba8ec94a38da700e2e80d 192.168.11.40:9007@19007 myself,master - 0 1620933900000 8 connected 0-1365 4096 5462-6826 10923-12287
...

檢查節(jié)點(diǎn)之間槽的均衡性

$ /usr/local/redis/bin/redis-cli --cluster rebalance 192.168.11.40:9001
...
[OK] All 16384 slots covered.
*** No rebalancing needed! All nodes are within the 2.00% threshold.

遷移之后所有主節(jié)點(diǎn)負(fù)責(zé)的槽數(shù)量差異在 2% 以?xún)?nèi)，因此集群節(jié)點(diǎn)數(shù)據(jù)相對(duì)均勻，無(wú)需調(diào)整

二、集群收縮

1. 遷移槽

執(zhí)行 reshard 三次，將數(shù)據(jù)平均分布到其他三個(gè)節(jié)點(diǎn)

2. 忘記節(jié)點(diǎn)

60s 內(nèi)對(duì)所有節(jié)點(diǎn)執(zhí)行如下操作：（不建議）

# 執(zhí)行后，會(huì)將該節(jié)點(diǎn)加入禁用列表（持續(xù) 60s），不再向其發(fā)送 Gossip 消息
cluster forget {nodeId}

建議使用 redis-cli 的 del-node 忘記節(jié)點(diǎn)：

/usr/local/redis/bin/redis-cli --cluster del-node {host:port} {nodeId}

內(nèi)部偽代碼

def delnode_cluster_cmd(downNode):
    # 下線節(jié)點(diǎn)不允許包含slots
    if downNode.slots.length != 0
        exit 1
    end
    # 向集群內(nèi)節(jié)點(diǎn)發(fā)送cluster forget
    for n in nodes:
        if n.id == downNode.id:
            # 不能對(duì)自己做forget操作
            continue;
        # 如果下線節(jié)點(diǎn)有從節(jié)點(diǎn)則把從節(jié)點(diǎn)指向其他主節(jié)點(diǎn)
        if n.replicate && n.replicate.nodeId == downNode.id :
            # 指向擁有最少?gòu)墓?jié)點(diǎn)的主節(jié)點(diǎn)
            master = get_master_with_least_replicas();
            n.cluster("replicate",master.nodeId);
        #發(fā)送忘記節(jié)點(diǎn)命令
        n.cluster('forget',downNode.id)
    # 節(jié)點(diǎn)關(guān)閉
    downNode.shutdown();

若主從節(jié)點(diǎn)都要下線，先下線從，避免全量復(fù)制

以上就是Redis 集群伸縮原理的詳細(xì)內(nèi)容，更多關(guān)于Redis 集群原理的資料請(qǐng)關(guān)注腳本之家其它相關(guān)文章！

您可能感興趣的文章: