docker swarm 集群搭建

实验环境采用 1个 master ，3个 work节点，4台机器组成的一个集群环境。

安装4台服务器的系统

学习的时候最好采用vm虚拟机，安装好一个初始化的可用的带docker的系统，
然后打一个快照。后续的学习系统都可以从这里克隆出来，非常快的。

安装docker环境

可以参考官方文档。

或者参考我这个文档 https://magesfc.github.io/mage/64aa5aff5028b66fde1c4d99fabd89113f42a31a/

初始化 docker swarm master 节点

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# docker swarm init --help  

Usage:  docker swarm init [OPTIONS]

Initialize a swarm

Options:
      --advertise-addr string                  Advertised address (format: "<ip|interface>[:port]")
      --autolock                               Enable manager autolocking (requiring an unlock key to start a stopped
                                               manager)
      --availability string                    Availability of the node ("active", "pause", "drain") (default "active")
      --cert-expiry duration                   Validity period for node certificates (ns|us|ms|s|m|h) (default 2160h0m0s)
      --data-path-addr string                  Address or interface to use for data path traffic (format: "<ip|interface>")
      --data-path-port uint32                  Port number to use for data path traffic (1024 - 49151). If no value is
                                               set or is set to 0, the default port (4789) is used.
      --default-addr-pool ipNetSlice           default address pool in CIDR format (default [])
      --default-addr-pool-mask-length uint32   default address pool subnet mask length (default 24)
      --dispatcher-heartbeat duration          Dispatcher heartbeat period (ns|us|ms|s|m|h) (default 5s)
      --external-ca external-ca                Specifications of one or more certificate signing endpoints
      --force-new-cluster                      Force create a new cluster from current state
      --listen-addr node-addr                  Listen address (format: "<ip|interface>[:port]") (default 0.0.0.0:2377)
      --max-snapshots uint                     Number of additional Raft snapshots to retain
      --snapshot-interval uint                 Number of log entries between Raft snapshots (default 10000)
      --task-history-limit int                 Task history retention limit (default 5)


执行 docker swarm init 带上 master 节点的 IP 地址

1
2
3
4
5
6
7
8
# docker swarm init --advertise-addr=192.168.1.112 
Swarm initialized: current node (oy7rs34fbjue71hq8k5gtlelo) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-1pq9zqbgvm6o50th0pxbywai971805s9fu4fiw4cnejn0jnzv6-aajxxekc40t7c1o6ofpt8rlgj 192.168.1.112:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions. # 如果要另外一个master节点加入，执行这个命令

执行完这个 init 之后又什么变化呢？

通过htop 查看并没有启动什么新的进程服务。

通过查看 /var/lib/docker 目录下面 发现 swarm 目录下面生成了新的文件了。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
cat docker-state.json 
{
  "LocalAddr":"192.168.1.112",
  "RemoteAddr":"",
  "ListenAddr":"0.0.0.0:2377",
  "AdvertiseAddr":"192.168.1.112:2377",
  "DataPathAddr":"",
  "DefaultAddressPool":null,
  "SubnetSize":24,
  "DataPathPort":4789,
  "JoinInProgress":false
}       

cat state.json
[{"node_id":"oy7rs34fbjue71hq8k5gtlelo","addr":"192.168.1.112:2377"}]


初始化 docker swarm work 节点

1
2
3
4

# docker swarm join --token SWMTKN-1-1pq9zqbgvm6o50th0pxbywai971805s9fu4fiw4cnejn0jnzv6-aajxxekc40t7c1o6ofpt8rlgj 192.168.1.112:2377
This node joined a swarm as a worker.  这个是打印出来的

加入完毕之后 可以 使用 ocker node ls 命令查看集群里面的服务器信息。

1
2
3
4
5
6
7
8

# docker node ls
ID                            HOSTNAME       STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
oy7rs34fbjue71hq8k5gtlelo *   swarm-master   Ready     Active         Leader           24.0.5              这台是master节点，hostname那个列显示了主机名。
5dcuxo3b2hhgf2rhlf3fpcpun     swarm-work1    Ready     Active                          24.0.5
i870enyo9md194jvwl73fnqfn     swarm-work2    Ready     Active                          24.0.5
69bet6xtvm9361w5cv2uac5xf     swarm-work3    Ready     Active                          24.0.5

其他，集群配置信息

1
2
3
4
5
6
7
8
9
10

# cat /etc/docker/daemon.json 
{
  "registry-mirrors": ["https://docker.mirrors.ustc.edu.cn/"],
  "log-driver": "journald",
  "exec-opts": [
    "native.cgroupdriver=systemd"
  ]
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
# docker info
Client: Docker Engine - Community
 Version:    24.0.5
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.20.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: journald
 Cgroup Driver: systemd
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: active
  NodeID: oy7rs34fbjue71hq8k5gtlelo
  Is Manager: true
  ClusterID: iny7tskll2a8n8ltkgq8quy2p
  Managers: 1
  Nodes: 4
  Default Address Pool: 10.0.0.0/8
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 192.168.1.112
  Manager Addresses:
   192.168.1.112:2377
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 5.4.250-1.el7.elrepo.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 3.806GiB
 Name: swarm-master
 ID: 893e34e6-e0a9-42e2-aa59-5e5c14ebf134
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Registry Mirrors:
  https://docker.mirrors.ustc.edu.cn/
 Live Restore Enabled: false