[Bug]: 配置milvus-cdc后，数据库名能同步，库里面的数据无法同步 #208
Description
Current Behavior
主机A（10.200.83.170）同步数据到主机B（10.200.83.171）
以下为主机A的milvus配置
version: '3.5'
services:
etcd:
container_name: milvus-etcd
image: quay.io/coreos/etcd:v3.5.5
environment:
- ETCD_AUTO_COMPACTION_MODE=revision
- ETCD_AUTO_COMPACTION_RETENTION=1000
- ETCD_QUOTA_BACKEND_BYTES=4294967296
- ETCD_SNAPSHOT_COUNT=50000
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/etcd:/etcd
command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd
healthcheck:
test: ["CMD", "etcdctl", "endpoint", "health"]
interval: 30s
timeout: 20s
retries: 3
ports:
- "2379:2379"
- "2380:2380"
pulsar:
container_name: pulsar_standalone
image: apachepulsar/pulsar:latest
logging:
driver: "json-file"
options:
max-size: "500m"
ports:
- "6650:6650"
- "8080:8080"
volumes:
- /etc/localtime:/etc/localtime:ro
command: bash -c "bin/pulsar standalone"
minio:
container_name: milvus-minio
image: minio/minio:RELEASE.2023-03-20T20-16-18Z
environment:
MINIO_ACCESS_KEY: minioadmin
MINIO_SECRET_KEY: minioadmin
ports:
- "9001:9001"
- "9000:9000"
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/minio:/minio_data
command: minio server /minio_data --console-address ":9001"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3
attu:
container_name: attu
image: zilliz/attu:latest
environment:
MILVUS_URL: milvus-standalone:19530
ports:
- "8000:3000"
depends_on:
- "standalone"
standalone:
container_name: milvus-standalone
image: milvusdb/milvus:v2.4.8
command: ["milvus", "run", "standalone"]
security_opt:
- seccomp:unconfined
environment:
ETCD_ENDPOINTS: etcd:2379
MINIO_ADDRESS: minio:9000
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/milvus:/var/lib/milvus
- /storage/milvus2.4.8/milvus.yaml:/milvus/configs/milvus.yaml:ro
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"]
interval: 30s
start_period: 90s
timeout: 20s
retries: 3
ports:
- "19530:19530"
- "9091:9091"
depends_on:
- "etcd"
- "minio"
networks:
default:
name: milvus
以下为milvus单机版的配置文件milvus.yaml修改部分
pulsar:
address: 10.200.83.170
port: 6650
容器能正常起来后配置milvus-cdc的docker-compose配置
version: '3'
services:
milvus-cdc:
container_name: milvus-cdc
image: milvus-cdc:latest
volumes:
- /etc/localtime:/etc/localtime:ro
- /storage/milvus-cdc/server/configs/cdc.yaml:/app/configs/cdc.yaml:ro
ports:
- "8444:8444"
networks:
default:
name: milvus-cdc
此处单独加载cdc.yaml配置。配置如下
address: 0.0.0.0:8444
maxTaskNum: 100
metaStoreConfig:
storeType: etcd
etcdEndpoints:
- 10.200.83.170:2379
etcd:
address:
- http://127.0.0.1:2379
enableAuth: false
username: root
password: root123456
enableTLS: false
tlsCertPath: deployment/cert/client.pem # path to your cert file
tlsKeyPath: deployment/cert/client.key # path to your key file
tlsCACertPath: deployment/cert/ca.pem # path to your CACert file
tlsMinVersion: 1.3
mysqlSourceUrl: root:root@tcp(127.0.0.1:3306)/milvuscdc?charset=utf8
rootPath: cdc
sourceConfig:
etcd:
address:
- http://10.200.83.170:2379
rootPath: by-dev
metaSubPath: meta
enableAuth: false
# username: root
#password: root123456
#enableTLS: false
#tlsCertPath: deployment/cert/client.pem # path to your cert file
#tlsKeyPath: deployment/cert/client.key # path to your key file
#tlsCACertPath: deployment/cert/ca.pem # path to your CACert file
#tlsMinVersion: 1.3
readChanLen: 4
defaultPartitionName: _default
replicateChan: by-dev-replicate-msg
pulsar:
address: pulsar://10.200.83.170:6650
webAddress: 10.200.83.170:80
maxMessageSize: 5242880
tenant: public
namespace: default
authPlugin: org.apache.pulsar.client.impl.auth.AuthenticationToken
authParams: token:xxx
kafka:
address: 127.0.0.1:9092
maxNameLength: 256
logLevel: info
detectDeadLock: false
milvus和milvus-cdc启动之后配置cdc任务，同时AB开启pulsar消息 170同步到171后，数据库名能同步，数据库内的数据无法同步，麻烦看下配置是哪里有问题？cdc日志如下：
Expected Behavior
No response
Steps To Reproduce
No response
Environment
No response
Anything else?
No response