k8s部署redis sentinel(哨兵)的实现

Dockerfile:

FROM redis:latest
WORKDIR /app
COPY *.conf conf/
COPY run.sh .
RUN chmod +x run.sh
CMD /app/run.sh

redis.conf:

bind 0.0.0.0
daemonize yes
logfile "/var/log/redis.log"
masterauth 123456
requirepass 123456
appendonly yes

sentinel.conf:

bind 0.0.0.0
port 26379
sentinel monitor redis-master redis-0.redis 6379 2
sentinel auth-pass redis-master 123456
sentinel down-after-milliseconds redis-master 10000
sentinel resolve-hostnames yes

sentinel resolve-hostnames 一定要显示开启,否则:

默认值是NO,sentinel不解析主机名,会把redis-0.redis当作IP处理——无法解析,这种情况在docker bridge网络、kubernetes service名称以及跨主机部署中极易出现。

run.sh:

#!/bin/bash
pod_seq=$(echo $POD_NAME |awk -F'-' '{print $2}')
if [[ $pod_seq -gt 0 ]];then
  sed -i '/^slaveof /d' /app/conf/redis.conf
  echo "slaveof redis-0.redis 6379" >>/app/conf/redis.conf
fi
redis-server /app/conf/redis.conf
sleep 15
redis-sentinel /app/conf/sentinel.conf &
tail -f /var/log/redis.log
docker build -t redis-sentinel:latest .
docker save redis-sentinel:latest |gzip -9 >redis-sentinel.img.gz
for i in node1 node2;do scp redis-sentinel.img.gz $i:/root;done
for i in node1 node2;do ssh $i "docker load -i /root/redis-sentinel.img.gz";done

redis-sentinel.yml:

apiVersion: v1
kind: Namespace
metadata:
  name: redis-ns
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis
  namespace: redis-ns
spec:
  serviceName: redis
  selector:
    matchLabels:
      app: redis
  replicas: 3
  template:
    metadata:
      labels:
        app: redis
    spec:
      nodeSelector:
        productLine: redis-ns
        area: wuhan
      restartPolicy: Always
      containers:
        - name: redis
          image: redis_sentinel:latest
          imagePullPolicy: IfNotPresent
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
          livenessProbe:
            tcpSocket:
              port: 6379
            initialDelaySeconds: 3
            periodSeconds: 5
          readinessProbe:
            tcpSocket:
              port: 6379
            initialDelaySeconds: 3
            periodSeconds: 5
          ports:
            - containerPort: 6379
          resources:
            requests:
              memory: 256Mi
              cpu: 50m
            limits:
              memory: 256Mi
              cpu: 200m
---
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: redis-ns
spec:
  clusterIP: None #无头服务,返回所有pod的IP
  ports:
    - name: redis
      port: 26379
      targetPort: 26379
  selector:
    app: redis


kubectl apply -f redis-sentinel.yml

k8s其它命令空间的java进程连接redis哨兵

一、核心原理

Redis 哨兵的工作机制是:客户端先连接哨兵集群获取主节点地址,再与主节点建立连接。在 K8s 中,跨命名空间访问需要:

  1. 为 Redis 哨兵集群创建无头服务(Headless Service)(保证能解析到所有哨兵 Pod 的 IP);
  2. Java 客户端配置哨兵地址时,使用 K8s 跨命名空间的服务域名;
  3. 确保命名空间之间的网络策略允许通信(如没有禁用跨命名空间访问)。

二、前提条件

  1. Redis 哨兵集群已部署在 K8s 中(假设命名空间为 redis-ns);
  2. Java 应用所在命名空间(如 app-ns)能访问 redis-ns 的 Pod/Service;
  3. Java 项目引入 Redis 客户端(以主流的 lettucejedis 为例,推荐 lettuce)。

Java 客户端配置(跨命名空间连接)

1、依赖引入(pom.yml)

<!-- Spring Data Redis 核心依赖 -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<!-- 如需使用 jedis,排除 lettuce 并引入 jedis -->
<dependency>
    <groupId>redis.clients</groupId>
    <artifactId>jedis</artifactId>
</dependency>
<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-pool2</artifactId>
</dependency>

2、配置文件(application.yml)

核心是 spring.redis.sentinel.nodes 配置为 K8s 跨命名空间的哨兵服务域名:

spring:
  redis:
    # Redis 密码(如果有)
    password: your-redis-password
    # 数据库索引(默认 0)
    database: 0
    # 哨兵配置
    sentinel:
      # 哨兵监控的主节点名称(必须和 Redis 哨兵配置的一致)
      master: mymaster
      # 跨命名空间的哨兵地址:服务名.命名空间.svc.cluster.local:端口
      # 多个哨兵用逗号分隔(无头服务会解析到所有哨兵 Pod IP)
      nodes: redis-sentinel.redis-ns.svc.cluster.local:26379
    # 连接池配置(按需调整)
    lettuce:
      pool:
        max-active: 8   # 最大连接数
        max-idle: 8     # 最大空闲连接
        min-idle: 0     # 最小空闲连接
        max-wait: -1ms  # 最大等待时间(-1 表示无限制)
    # 如果用 jedis,替换 lettuce 为 jedis 即可
    # jedis:
    #   pool:
    #     max-active: 8
    #     max-idle: 8
    #     min-idle: 0
    #     max-wait: -1ms

如果是纯 Java 项目(无 Spring Boot),以 lettuce 为例:

import io.lettuce.core.RedisClient;
import io.lettuce.core.RedisURI;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.api.sync.RedisCommands;
import io.lettuce.core.sentinel.api.StatefulRedisSentinelConnection;

public class RedisSentinelClient {
    public static void main(String[] args) {
        // 1. 配置哨兵地址(跨命名空间域名)
        RedisURI sentinelUri = RedisURI.builder()
                .withSentinel("redis-sentinel.redis-ns.svc.cluster.local", 26379)
                .withSentinelMasterId("redis-master")  // 哨兵监控的主节点名称
                .withPassword("123456".toCharArray())  // Redis 密码
                .withDatabase(0)
                .build();

        // 2. 创建哨兵客户端
        RedisClient client = RedisClient.create(sentinelUri);
        StatefulRedisSentinelConnection<String, String> sentinelConnection = client.connectSentinel();
        
        // 3. 获取主节点连接
        StatefulRedisConnection<String, String> connection = sentinelConnection.master("redis-master");
        RedisCommands<String, String> commands = connection.sync();

        // 4. 测试连接
        try {
            String result = commands.set("test-key", "hello-k8s-redis-sentinel");
            System.out.println("Set 结果:" + result);
            System.out.println("Get 结果:" + commands.get("test-key"));
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            // 5. 关闭连接
            connection.close();
            sentinelConnection.close();
            client.shutdown();
        }
    }
}
Categories: docker与kubernetes