因为部分核心业务需要实现双活的架构,原来架构也是使用redis哨兵集群,为了解决双活业务系统的数据共享问题,利用阿里云主机搭建了公网redis哨兵集群。同时考虑到官网的cluster模式需要的主机成本过高,没有选用。
一、环境说明
北京 2核4g 低配主机(根据自己业务评估数据量)
深圳 2核4g
杭州 2核4g
三个区域创建独立的主机,vpc网络,安全组
架构: 一主两从三哨兵
二、配置过程
2.1 安全组配置
1、三地安全组开通互访权限,针对IP点对点开放(一定要慎重,控制开放范围,redis一定要禁止外部公开访问)
2、禁ping策略
3、ssh修改默认端口,密码策略要严格,有条件可以上堡垒机
三、安装redis (3个节点都执行)
yum install epel-release
yum redis
四、配置redis
server(主)
pidfile "/var/run/redis.pid" logfile "/var/log/redis/redis.log" daemonize yes port 6379 tcp-backlog 511 timeout 0 tcp-keepalive 0 loglevel notice databases 16 save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename "dump.rdb" dir "/var/lib/redis" slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 masterauth "yourpassword" requirepass "yourpassword" maxmemory-policy volatile-lru appendonly no appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes lua-time-limit 5000 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "xE" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-entries 512 list-max-ziplist-value 64 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yesserver从
pidfile "/var/run/redis.pid" logfile "/var/log/redis/redis.log" daemonize yes port 6379 tcp-backlog 511 timeout 0 tcp-keepalive 0 loglevel notice databases 16 save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename "dump.rdb" dir "/var/lib/redis" slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 masterauth "yourpassword" requirepass "yourpassword" maxmemory-policy volatile-lru appendonly no appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes lua-time-limit 5000 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "xE" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-entries 512 list-max-ziplist-value 64 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes # Generated by CONFIG REWRITE supervised systemd slaveof 主公网IP 6379另外一个从库根据上面配置扩展,也可以多个从库
下面主要说下哨兵的部署
port 26379 dir "/tmp" daemonize yes protected-mode no #sentinel心跳检测主3秒内无响应,视为挂掉,开始切换其他从为主 sentinel myid d38c479d37e5316cc29ebc1c6dbfe5e24a55a3ad logfile "/var/log/redis/sentinel.log" sentinel monitor mymaster reids主库公网IP 6379 2 sentinel auth-pass mymaster 'yourpassword' sentinel announce-ip "本机公网映射IP" sentinel announce-port 26379阿里云vpc网络一般公网ip采用映射的方式到ECS上面,以下配置是关键
sentinel announce-ip "本机公网映射IP" sentinel announce-port 26379
sentinel-announce-ip 一定要配置,不然sentinel之间通信使用的是内网地址,无法通信,导致集群通信异常,master故障切换会失败。
其它两个哨兵类似配置,不再累述。
启动集群(三个节点)
systemctl enable redis.service
systemctl start redis.service
systemctl enable redis-sentinel systemctl start redis-sentinel
五、养成好习惯,部署好验证下
redis-cli -p 26379
127.0.0.1:26379> info
# Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mymaster,status=ok,address=公网IP:6379,slaves=2,sentinels=3
后记,一定要做好安全防御,redis一旦被期权