C/C++编译时经常需要配置环境，有的时候不想安装在本机上，环境太多容易混乱，安装步骤麻烦，出错也不好撤销修改。我最近在学习Linux C/C++，但是我的电脑是win10系统，正好可以结合vscode+VMware+Docker+SSH做出一套Linux下面的C/C++编译环境，配置方便快捷安装软件及功能列表gccg++gdbmakemanpsmisc(killall命令)openssh-servervsftpd(FTP服务)编写Dockerfile文件进入到~目录，创建cpp_env文件夹，创建Dockerfile文件cd ~ mkdir cpp_dev cd cpp_dev vi Dockerfile粘贴以下内容脚本中的developer为账号脚本中的password为密码# 使用最新的Ubuntu镜像作为基础镜像 FROM ubuntu:22.04 # 禁用SELinux ENV container SELINUX=disabled # 更换为阿里云的Ubuntu存储库镜像，以加速更新 RUN sed -i 's/http:\/\/archive.ubuntu.com/http:\/\/mirrors.aliyun.com/g' /etc/apt/sources.list # 创建SSH目录并更新apt包列表 RUN mkdir /run/sshd && \ apt-get update && \ apt-get install -y gcc g++ gdb make man psmisc openssh-server vsftpd && \ apt-get clean && \ rm -rf /var/lib/apt/lists/* # 创建一个新用户（这里使用了用户名为bill，您可以更改为您喜欢的用户名） RUN useradd -m -s /bin/bash bill && \ echo 'bill:bill' | chpasswd && \ usermod -aG sudo bill # 允许SSH远程连接 RUN sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin no/' /etc/ssh/sshd_config && \ sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/' /etc/ssh/sshd_config # 配置vsftpd RUN echo "local_enable=YES" >> /etc/vsftpd.conf && \ echo "write_enable=YES" >> /etc/vsftpd.conf && \ echo "local_umask=022" >> /etc/vsftpd.conf && \ echo "chroot_local_user=YES" >> /etc/vsftpd.conf && \ echo "pasv_enable=YES" >> /etc/vsftpd.conf && \ echo "pasv_min_port=30000" >> /etc/vsftpd.conf && \ echo "pasv_max_port=30009" >> /etc/vsftpd.conf && \ echo "allow_writeable_chroot=YES" >> /etc/vsftpd.conf && \ echo "local_root=/home/bill/ftp" >> /etc/vsftpd.conf # 创建FTP所需目录并设置权限 RUN mkdir -p /var/run/vsftpd/empty && \ chmod 555 /var/run/vsftpd/empty # 允许外部连接 EXPOSE 22 21 30000-30009 # 启动SSH和FTP服务器 CMD ["/bin/bash", "-c", "/usr/sbin/sshd && /usr/sbin/vsftpd /etc/vsftpd.conf"]打包成Docker镜像sudo docker build -t my_cpp_env .运行容器所有容器的端口号均设置2200的偏移量，例如SSH端口号为2222，避免与宿主机冲突设置启动参数--restart always，容器总是随着主机启动而启动-v /home/bill:/home/bill 将本地主机的 /home/bill 文件夹映射到容器内的 /home/bill 文件夹容器的22端口（SSH端口）映射到主机的22端口，将20和21端口（FTP端口）映射到主机的20和21端口，并将30000到30009端口（FTP被动模式端口）映射到主机的30000到30009端口（只用主动模式操作）sudo docker run -d \ -v /home/bill:/home/bill \ -p 2222:22 \ -p 20-21:20-21 \ --restart always \ --name my_cpp_container \ my_cpp_env查看容器日志sudo docker logs -f my_cpp_container删除容器# 先停止容器再删除 sudo docker container stop my_cpp_container sudo docker container rm my_cpp_container # 或者强制删除 sudo docker container rm -f my_cpp_container查看容器运行状态[bill@localhost cppenv]$ sudo docker container ls CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 205356368e4f my_cpp_env "/usr/sbin/sshd -D" 36 minutes ago Up 36 minutes 0.0.0.0:2222->22/tcp, :::2222->22/tcp my_cpp_container容器正常启动，随后可以通过SSH连接。容器开机自启在容器运行参数中设置启动参数--restart always，容器总是随着主机启动而启动设置docker开机自启sudo systemctl enable docker配置vscode远程连接安装Remote - SSH插件插件商店搜索SSH关键字，找到如下插件，进行安装安装成功后，左侧功能栏会有如下图标：添加SSH远程连接点击+添加SSH连接在Enter SSH Connection Command连接命令文本框中输入# 用户名、IP地址、端口号改成自己定义的即可 ssh bill@192.168.149.137 -p 2222在弹出的custom SSH configuration file中选择第一个备选项即可，这个配置文件存到了当前用户的.ssh文件夹中Host 192.168.149.138(Docker) HostName 192.168.149.138 User bill Port 2222连接点击➡小箭头进行连接连接成功测试编译编译成功，环境配置完成
我使用的服务器集群架构是三节点Docker Swarm，一主二从。基于Percona XtraDB Cluster（PXC），以及haproxy+keepalived配置了MySQL高可用架构，当然也是三节点的MySQL。今天我通过Haproxy面板，看到第三台服务器的MySQL离线了（pxc03），于是登录到服务器上查看容器的运行日志，别的英文没细看，一眼就看到了no space left，无存储空间了，df -h查看目录占用百分比100%，显然是/目录的磁盘空间满了，下一步就是排查是哪些程序占用了大部分磁盘。linux怎么查看文件夹多大1、利用ls命令查看，该命令可将结果以KB、MB等为单位进行显示，语法为ls -ll或ls-lh；2、利用du -h –max-depth=1 *命令，该命令可查看当前目录下的各个文件和文件夹的大小；3、利用du -sh命令，可查看当前文件夹的总大小；4、利用du -h –max-depth=0 *命令，可查看直接子目录文件及文件夹大小。通过查看是/var/lib/docker占用空间过大，通过进一步查看是/var/lib/docker/overlay2目录占用空间大，高达12GB。我之前在另外一台服务器上遇到了这种情况，当时我的解决方案是把安装在根目录的docker迁移到home下，因为home挂在的磁盘容量较大，但是这种做法是错误的，治标不治本，即使是迁移了docker安装目录，但是也没有把占用空间大的问题解决，我们需要寻找问题的根本原因，从根本解决这个问题。第一种情况：是因为docker中部署的系统中日志内容的不断扩大。这种情况下，我们可手动，或定时任务清理。对于/var/lib/docker/overlay2 空间占用，存在很多误导的方法是去迁移路径等。。其实磁盘空间的占用和overlay没关系，它的使用和真实的disk使用相同，overlay只是一个docker的虚拟文件系统，真实的文件系统是前者/dev/vda1,可以看到路径所指为根目录。所以，通过该目录去查找哪里占用资源过大。通常情况下与overlay2的同级目录下会有如下的目录：占用大量空间的日志文件位于containers下，该目录下存在如下目录：在该目录下，会存在以目录名为前缀，以“-json.log”为后缀的目录文件。例如：使用如下命令会看到该文件的大小：ll -h然后，使用> *-json.log，将该文件进行清理清理完成可不重启服务，重启更好清理完后，可通过df -h查看磁盘占用情况第二种情况：可通过执行docker system prune命令可用于清理磁盘，删除关闭的容器、无用的数据卷和网络，以及dangling镜像（即无tag的镜像），我的情况就是这种情况，docker中存在大量废弃的镜像docker system pruneDocker中查询磁盘占用情况：这个图片是清理后的截图，清理之前多出来10G+，忘记截图了docker system prune -a【慎用】命令清理更干净、彻底，可以将没有容器使用的Docker镜像都删掉。也可以使用Docker的图形化管理工具——Portainer进行查看清理参考资料Docker篇之Overlay2磁盘空间占用过大清理
Docker集群部署MySQL、Redis集群实例简介需要具备的基础MySQL数据库开发能力了解Redis，会简单操作了解Linux指令，能在Linux上安装卸载软件哪里需要集群？前后端分离部署图项目Demorenrenfast开源前后端项目Linux基础强化firewalld防火墙centos7默认安装了firewalld防火墙通过防火墙允许或限制数据的通过管理防火墙查看状态firewall-cmd --state启动、关闭、重启防火墙service firewall start service firewall stop service firewall restart端口管理# 开放 firewall-cmd --permanent --add-port=8080-8085/tcp # 重新加载防火墙配置 firewall-cmd --reload # 关闭端口，要与之前写的add时开放的端口范围相同，不能开放多个端口，只关闭一个端口 firewall-cmd --permanent --remove-port=8080-8085/tcp查看开启的端口firewall-cmd --permanent --list-ports查看使用网络的服务firewall-cmd --permanent --list-services云计算中的DockerDocker容器管理命令Docker镜像导入导出docker save java > /home/java.tar.gz docker load java < /home/java.tar.gzDocker暂停 停止容器docker pause container-name docker unpause container-name docker stop container-name docker start -i container-nameMySQL集群搭建常见MySQL集群方案选用PXC集群方式。PXC原理PXC（Percona XtraDB Cluster）任何一个节点都可读可写建议PXC使用Percona Server（MySQL改进版，性能提升很大）PXC与Replication方案对比PXC的数据强一致性PXC同步复制，事务在所有集群节点要么同时提交，要么不提交Replication采用异步复制，无法保证数据一致性PXC集群安装最新的PXC版本是PXC8.0，即对应MySQL8.0。本实验使用PXC5.7，即对应MySQL5.7，主要是因为大部分项目都采用的MySQL5.7 实验版本最新版本PXC5.78.0XtraBackup2.48.0 节点分配物理机IP: 10.248.192.137Docker内部网段子网掩码：172.18.0.0/24节点docker内部IP映射物理机端口MySQL-node1172.17.0.23306MySQL-node2172.17.0.33307MySQL-node3172.17.0.43308MySQL-node4172.17.0.53309MySQL-node4172.17.0.63310Haproxy172.17.0.74001 WEB看板端口 4002 MySQL负载均衡入口拉取Docker镜像docker pull percona/percona-xtradb-cluster docker image ls由于镜像的名字太长，不便于使用，我们可以在本地对镜像名称进行修改docker tag percona/percona-xtradb-cluster pxc docker image ls此时可以把原来的镜像删除掉docker iamge rm percona/percona-xtradb-cluster pxc创建内部网络出于安全考虑，需要给PXC集群实例创建Docker内部网络# 创建网段 docker network create net1 # 查看网段信息 docker network inspect net1 # 删除网段 docker network rm net1docker默认的网段是172.17.0.x创建网络的时候不指定网段，会默认从dokcer默认网段向下排，例如下一个网段是172.18.0.x创建指定网段docker network create --subnet=172.18.0.0/24 net1 docker inspect net1 # 如果删除 docker network rm net1子网掩码172.18.0.0/24中的/24的意思是172.17.0前三位固定，最后一位动，即子网掩码为255.255.255.0，用二进制表示前面就是24位1如果子网掩码是172.18.0.0/16，那么172.18固定，后面两个段动。创建Docker卷业务数据通常存在物理机里，而不是存在容器内PXC无法使用目录映射方式运行（启动会闪退），所以需要使用Docker卷技术# --name可以省略不写 docker volume create --name v1 docker inspect v1 [ { "CreatedAt": "2022-08-18T09:41:14+08:00", "Driver": "local", "Labels": {}, "Mountpoint": "/var/snap/docker/common/var-lib-docker/volumes/v1/_data", "Name": "v1", "Options": {}, "Scope": "local" } ] # 如果删除 docker volume rm v1volume在物理机的/var/snap/docker/common/var-lib-docker/volumes/v1/_data位置创建PXC容器向PXC镜像传入运行参数就能创建出PXC容器MYSQL_ROOT_PASSWORD 数据库密码，账户默认为rootCLUSTER_NAME 集群名称XTRABACKUP_PASSWORD 数据库节点之间同步的密码--privileged：使用该参数，container内的root拥有真正的root权限使用pxc镜像第一个启动的PXC节点是主节点，它要初始化PXC集群PXC启动之后，就没有主节点角色了PXC集群任何一个节点都是可以读写数据的# 第一个PXC docker run -d -p 3306:3306 \ -e MYSQL_ROOT_PASSWORD=abc123456 \ -e CLUSTER_NAME=PXC \ -e XTRABACKUP_PASSWORD=abc123456 \ -v v1:/var/lib/mysql \ --privileged \ --name=node1 \ --net=net1 --ip=172.18.0.2 \ pxc # 创建第2、3、4、5个volume # docker volume create --name v1 上面创建过了 docker volume create --name v2 docker volume create --name v3 docker volume create --name v4 docker volume create --name v5 # 创建其他4个节点 # 需要修改端口、挂载卷名、IP、容器名，同时增加CLUSTER_JOIN参数 # 第2个PXC docker run -d -p 3307:3306 \ -e MYSQL_ROOT_PASSWORD=abc123456 \ -e CLUSTER_NAME=PXC \ -e XTRABACKUP_PASSWORD=abc123456 \ -e CLUSTER_JOIN=node1 \ -v v2:/var/lib/mysql \ --privileged --name=node2 \ --net=net1 --ip=172.18.0.3 \ pxc # 第3个PXC docker run -d -p 3308:3306 -e CLUSTER_NAME=PXC -e MYSQL_ROOT_PASSWORD=abc123456 -e XTRABACKUP_PASSWORD=abc123456 -e CLUSTER_JOIN=node1 -v v3:/var/lib/mysql --privileged --name=node3 --net=net1 --ip=172.18.0.4 pxc # 第4个PXC docker run -d -p 3309:3306 -e CLUSTER_NAME=PXC -e MYSQL_ROOT_PASSWORD=abc123456 -e XTRABACKUP_PASSWORD=abc123456 -e CLUSTER_JOIN=node1 -v v4:/var/lib/mysql --privileged --name=node4 --net=net1 --ip=172.18.0.5 pxc # 第5个PXC docker run -d -p 3310:3306 -e CLUSTER_NAME=PXC -e MYSQL_ROOT_PASSWORD=abc123456 -e XTRABACKUP_PASSWORD=abc123456 -e CLUSTER_JOIN=node1 -v v5:/var/lib/mysql --privileged --name=node5 --net=net1 --ip=172.18.0.6 pxc必须主节点可以访问了，才能创建从节点，否则会闪退docker container ls CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 6563dac8b2ae pxc "/entrypoint.sh mysq…" 6 seconds ago Up 6 seconds 4567-4568/tcp, 33060/tcp, 0.0.0.0:3310->3306/tcp, :::3310->3306/tcp node5 17c1d1314252 pxc "/entrypoint.sh mysq…" 8 seconds ago Up 8 seconds 4567-4568/tcp, 33060/tcp, 0.0.0.0:3309->3306/tcp, :::3309->3306/tcp node4 e58f6800f23d pxc "/entrypoint.sh mysq…" 9 seconds ago Up 8 seconds 4567-4568/tcp, 33060/tcp, 0.0.0.0:3308->3306/tcp, :::3308->3306/tcp node3 45df9d0864fb pxc "/entrypoint.sh mysq…" 26 seconds ago Up 26 seconds 4567-4568/tcp, 33060/tcp, 0.0.0.0:3307->3306/tcp, :::3307->3306/tcp node2 893961f0c9e3 pxc "/entrypoint.sh mysq…" 14 minutes ago Up 14 minutes 4567-4568/tcp, 0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp node1node2-node5闪退问题(PXC8.0)上面的步骤操作完后，过一会儿node2-node5会闪退。本实验使用的是PXC8.0（Percona Server for MySQL 8.0），直接启动node2-5会出现MySQL无法启动的情况。查看log信息大致如下：2022-08-18T06:24:23.078258Z 0 [ERROR] [MY-000000] [Galera] failed to open gcomm backend connection: 110: failed to reach primary view (pc.wait_prim_timeout): 110 (Connection timed out) at gcomm/src/pc.cpp:connect():161 2022-08-18T06:24:23.078271Z 0 [ERROR] [MY-000000] [Galera] gcs/src/gcs_core.cpp:gcs_core_open():219: Failed to open backend connection: -110 (Connection timed out) 2022-08-18T06:24:24.078556Z 0 [Note] [MY-000000] [Galera] gcomm: terminating thread 2022-08-18T06:24:24.078655Z 0 [Note] [MY-000000] [Galera] gcomm: joining thread 2022-08-18T06:24:24.078865Z 0 [ERROR] [MY-000000] [Galera] gcs/src/gcs.cpp:gcs_open():1811: Failed to open channel 'PXC' at 'gcomm://node1': -110 (Connection timed out) 2022-08-18T06:24:24.078959Z 0 [ERROR] [MY-000000] [Galera] gcs connect failed: Connection timed out 2022-08-18T06:24:24.079004Z 0 [ERROR] [MY-000000] [WSREP] Provider/Node (gcomm://node1) failed to establish connection with cluster (reason: 7) 2022-08-18T06:24:24.079052Z 0 [ERROR] [MY-010119] [Server] Aborting 2022-08-18T06:24:24.079551Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.28-19.1) Percona XtraDB Cluster (GPL), Release rel19, Revision f544540, WSREP version 26.4.3.虽说报错信息和文章中的不一样，但是确实是我在参考其他资料时没有提到的一点，一个变量：pxc-encrypt-cluster-traffic上，根据官网资料，pxc-encrypt-cluster-traffic 变量默认是启用，所以要求所有节点使用相同的密钥和证书文件，MySQL生成默认密钥和证书文件，并将其放在数据目录中。这些自动生成的文件适用于自动 SSL 配置。所以将第一个节点数据目录下的 *.pem全部复制到第二个节点，再次执行systemctl start mysql命令，节点正常启动。所以，通过docker inspect node1查看v1卷的物理路径，然后复制其证书文件到v2、v3、v4、v5卷中即可。cp *.pem /var/snap/docker/common/var-lib-docker/volumes/v2/_data/ cp *.pem /var/snap/docker/common/var-lib-docker/volumes/v3/_data/ cp *.pem /var/snap/docker/common/var-lib-docker/volumes/v4/_data/ cp *.pem /var/snap/docker/common/var-lib-docker/volumes/v5/_data/再重启node2-node5的容器即可docker container stop node2 node3 node4 node5 docker container start node2 node3 node4 node5数据库负载均衡虽然搭建了集群，但是没有负载均衡，请求还是只会落在一个数据库上使用Haproxy做负载均衡，请求会均匀分发给每个PXC节点，单节点负载低，性能好负载均衡中间件对比安装Haproxydocker pull haproxy docker image ls创建Haproxy配置文件Haproxy配置文件需要自己写touch /home/bill/haproxy/haproxy.cfg配置模板global #工作目录 chroot /usr/local/etc/haproxy #日志文件，使用rsyslog服务中local5日志设备（/var/log/local5），等级info log 127.0.0.1 local5 info #守护进程运行 daemon defaults log global mode http #日志格式 option httplog #日志中不记录负载均衡的心跳检测记录 option dontlognull #连接超时（毫秒） timeout connect 5000 #客户端超时（毫秒） timeout client 50000 #服务器超时（毫秒） timeout server 50000 #监控界面 listen admin_stats #监控界面的访问的IP和端口 bind 0.0.0.0:8888 #访问协议 mode http #URI相对地址 stats uri /dbs #统计报告格式 stats realm Global\ statistics #登陆帐户信息 stats auth admin:abc123456 #数据库负载均衡 listen proxy-mysql #访问的IP和端口 bind 0.0.0.0:3306 #网络协议 mode tcp #负载均衡算法（轮询算法） #轮询算法：roundrobin #权重算法：static-rr #最少连接算法：leastconn #请求源IP算法：source balance roundrobin #日志格式 option tcplog #在MySQL中创建一个没有权限的haproxy用户，密码为空。Haproxy使用这个账户对MySQL数据库心跳检测 option mysql-check user haproxy server MySQL_1 172.18.0.2:3306 check weight 1 maxconn 2000 server MySQL_2 172.18.0.3:3306 check weight 1 maxconn 2000 server MySQL_3 172.18.0.4:3306 check weight 1 maxconn 2000 server MySQL_4 172.18.0.5:3306 check weight 1 maxconn 2000 server MySQL_5 172.18.0.6:3306 check weight 1 maxconn 2000 #使用keepalive检测死链 option tcpka 第50-55行为心跳检测，需要在每个MySQL节点上创建一个无任何权限、无密码的haproxy用户，用来进行心跳检测# 在node1-node5上面执行 CREATE USER 'haproxy'@'%' IDENTIFIED BY ''; # 账号为haproxy，密码为空创建Haproxy容器8888是haproxy状态网页的端口3306的haproxy的默认端口，由于已经被node1占用，故需要修改映射端口考虑到高可用，至少2个Haproxy容器docker run -it -d \ -p 4001:8888 -p 4002:3306 \ -v /home/bill/haproxy:/usr/local/etc/haproxy \ --name h1 --privileged --net=net1 --ip=172.18.0.7 haproxy启动之后发现闪退，查看log如下：docker logs h1 [NOTICE] (1) : New worker (9) forked [NOTICE] (1) : Loading success. [NOTICE] (9) : haproxy version is 2.6.2-16a3646 [NOTICE] (9) : path to executable is /usr/local/sbin/haproxy [ALERT] (9) : [haproxy.main()] Cannot chroot(/usr/local/etc/haproxy). [NOTICE] (1) : haproxy version is 2.6.2-16a3646 [NOTICE] (1) : path to executable is /usr/local/sbin/haproxy [WARNING] (1) : Current worker (9) exited with code 1 (Exit) [ALERT] (1) : exit-on-failure: killing every processes with SIGTERM [WARNING] (1) : All workers exited. Exiting... (1)看到有Cannot chroot(/usr/local/etc/haproxy)推测权限问题，于是以root账户运行容器（加上-u 0参数）：docker run -it -d \ -u 0 \ -p 4001:8888 -p 4002:3306 \ -v /home/bill/haproxy:/usr/local/etc/haproxy \ --name h1 --privileged --net=net1 --ip=172.18.0.7 haproxy启动成功docker logs -f h1 [NOTICE] (1) : New worker (9) forked [NOTICE] (1) : Loading success.访问 http://10.248.192.137:4001/dbs ，输入账号密码（admin:abc123456），界面如下：再数据库连接工具中，输入4002端口连接：MySQL连接成功，Haproxy启动成功例如在H1中创建test数据库：create database test;可以看到node1-node5节点均同步新建了test数据库：Haproxy双机热备单节点Haproxy不具备高可用，必须要有冗余设计虚拟IPHaproxy双机热备方案利用Keepalived实现双机热备安装KeepalivedKeepalived必须安装在Haproxy所在的容器内apt-get update apt-get install keepalivedKeepalived配置文件Keepalived的配置文件路径是: /etc/keepalived/keepalived.confstate: Keepalived的身份（MASTER主服务，BACKUP备用服务器）。主服务抢占虚拟IP，备用服务不会抢占IPinterface: 宿主机的网卡名virtual_router_id: 虚拟路由ID，MASTER和BACKUP需要一致，值为0~255之间priority：MASTER的权重要高于BACKUP，根据硬件配置调节，数字越大优先级越高advert_int: 心跳检测时间间隔（秒）authentication: 主从服务的鉴权账户，主备服务需配置相同virtual_ipaddress: 虚拟IP地址。可以设置多个虚拟IP，每行一个vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 123456 } virtual_ipaddress { 172.18.0.201 } }haproxy容器内默认是没有vim编辑器的，需要手动安装vim编辑器apt-get install vim vi /etc/keepalived/keepalived.conf启动Keepalived服务service keepalived start在宿主机中验证配置ping 172.18.0.201 PING 172.18.0.201 (172.18.0.201) 56(84) bytes of data. 64 字节，来自 172.18.0.201: icmp_seq=1 ttl=64 时间=0.124 毫秒 64 字节，来自 172.18.0.201: icmp_seq=2 ttl=64 时间=0.085 毫秒 64 字节，来自 172.18.0.201: icmp_seq=3 ttl=64 时间=0.088 毫秒 64 字节，来自 172.18.0.201: icmp_seq=4 ttl=64 时间=0.087 毫秒 64 字节，来自 172.18.0.201: icmp_seq=5 ttl=64 时间=0.087 毫秒 ^C --- 172.18.0.201 ping 统计 --- 已发送 5 个包， 已接收 5 个包, 0% 包丢失, 耗时 4095 毫秒 rtt min/avg/max/mdev = 0.085/0.094/0.124/0.014 ms配置MASTER节点成功。MySQL的热备份冷备份与热备份冷备份冷备份需要关闭数据库，在进行备份操作，通常做法是拷贝数据文件冷备份是最简单最安全的一种备份方式大型网站无法做到关闭业务备份数据，所以冷备份不是最佳选择热备份在系统运行的状态下去备份，也是难度最大的备份MySQL常见的热备份有LVM和XtraBackup两种方案建议使用XtraBackup热备MySQLXtraBackup备份过程中不锁表，快速可靠XtraBackup备份过程中不会打断正在执行的任务（例如增删改查）XtraBackup能够基于压缩等功能节约磁盘空间和流量全量备份和增量备份全量备份是备份全部数据，备份过程时间长，占用空间大增量备份只备份变化的那部分数据。备份时间短，占用空间小增量备份操作步骤这部分不要操作，只是一个踩坑的地方。往下看单独部署XtraBackup容器先创建一个volume，用来存放备份的数据docker volume create backup把volume映射到node1上docker stop node1node1节点成功关闭，下面需要把node1删掉，重新创建node1并挂载backup数据卷。由于node1的数据存储在v1卷，因此重新创建node1不影响节点的数据。# 删除node1节点 docker container rm node1 # 创建node1 docker run -d -p 3306:3306 \ -e MYSQL_ROOT_PASSWORD=abc123456 \ -e CLUSTER_NAME=PXC \ -e XTRABACKUP_PASSWORD=abc123456 \ -e CLUSTER_JOIN=node2 \ -v v1:/var/lib/mysql \ -v backup:/data \ --privileged \ --name=node1 \ --net=net1 --ip=172.18.0.2 \ pxc增加了-v backup:/data，映射备份目录增加了-e CLUSTER_JOIN=node2，加入到集群内，join到node2-node5任意一个节点均可node1节点成功启动。下一步进入到PXC容器安装XtraBackup，并执行备份：# 需要使用root身份操作 docker exec -it -u 0 node1 bash # PXC5.7镜像自带innobackupex # apt-get update # apt-get install percona-xtrabackup-24 # 全量备份 innobackupex --user=root --password=abc123456 /data/backup/full这里有个问题，pxc8.0的容器里没有apt-get命令，通过离线安装包方式安装也比较麻烦，不利于自动化构建。通过查阅资料，发现一种更好的XtraBackup部署方法：https://docs.percona.com/percona-xtrabackup/8.0/installation/docker.html根据官网教程，如果是PXC8.0，可以单独部署一个XtraBackup容器，在启动XtraBackup容器的时候，指定需要备份的MySQL节点，挂载需要备份的MySQL的volume，相比于上面的进到PXC容器去手动安装XtraBackup，要更方便、快捷PXC5.7镜像自带innobackupex，不用apt安装，而且PXC5.7容器内没有apt-getPXC全量恢复数据库可以热备份，但是不能热还原。只能新建空白的MySQL节点，再进行数据恢复，再建立PXC集群还原数据前要将未提交的事务回滚，还原数据之后重启MySQL查看数据在物理机的位置ls /var/snap/docker/common/var-lib-docker/volumes/backup/_data/backup/full/ 2022-08-23_04-34-13恢复数据操作# 删除MySQL集群及数据卷 docker container stop node1 node2 node3 node4 node5 docker container rm node1 node2 node3 node4 node5 docker volume rm v1 v2 v3 v4 v5 # 重新创建node1 docker volume create v1 docker run -d -p 3306:3306 \ -e MYSQL_ROOT_PASSWORD=abc123456 \ -e CLUSTER_NAME=PXC \ -e XTRABACKUP_PASSWORD=abc123456 \ -v v1:/var/lib/mysql \ -v backup:/data \ --privileged \ --name=node1 \ --net=net1 --ip=172.18.0.2 \ pxc # 进入到node1节点内 docker exec -it -u 0 node1 bash # ===================================== # 清空MySQL数据 rm -rf /var/lib/mysql/* # 未提交的事务回滚 innobackupex --user=root --password=abc123456 --apply-back /data/backup/full/2022-08-23_04-34-13 # 执行冷还原 innobackupex --user=root --password=abc123456 --copy-back /data/backup/full/2022-08-23_04-34-13 # 需要把MySQL数据权限改成mysql:mysql（默认为root:root，否则容器会重启失败） chown -R mysql:mysql /var/lib/mysql # 退出容器 exit # ===================================== # 重启容器 docker container restart node1MySQL_1节点启动成功，并且数据恢复成功（haproxy的心跳监测靠的是haproxy账号不断地去尝试登录MySQL）PXC集群搭建-生产环境所遇问题1. 不能映射目录，只能挂volumepath in container:/var/lib/mysql/var/log/mysql/backup2. 第一个创建的节点重启会启动失败参考CSDN：https://blog.csdn.net/qq_26896085/article/details/104547257我的解决方案是删掉容器（不要动创建好的volume），重新创建容器并添加启动参数-e CLUSTER_JOIN=pxc02，也就是加入到一个可用节点里3. 关于MySQL账户的恢复MySQL账户不能单纯的通过导入mysql数据库内的user和db表解决，单纯的导入这两个表的数据，MySQL节点之间会不同步。如果可以通过create user创建最好，可以自动同步。https://www.jb51.net/article/194392.htmmysqlpump -uroot -proot --exclude-databases=% --users --exclude-users=mysql.session,mysql.sys,root > /backup/user.sqlRedisCluster集群高速缓存利用内存保护数据，读写速度远超硬盘高速缓存可以减少I/O操作，降低I/O压力Redis是VMware开发的开源免费KV型NoSQL缓存产品Redis具有良好性能，最多可以提供10万次/秒的读写目前新浪微博团队组建了世界上最大的Redis集群（2018）Redis集群介绍RedisCluster: 官方推荐，没有中心节点（PXC也无中心节点）Codis: 中间件产品，存在中心节点（360公司推出）Twemproxy：中间件产品，存在中心节点使用RedisCluster无中心节点，客户端与redis节点直连，不需要中间代理层数据可以被分片存储（每个节点存储的数据不同）。每个节点有加冗余节点，节点故障可切换到冗余节点，保障集群正常运转管理方便，后续可自行增加或删除节点本实验搭建的是redis-6节点（一主一从结构）编号IP对外开放端口redis节点名1172.19.0.25001r12172.19.0.35002r23172.19.0.45003r34172.19.0.55004r45172.19.0.65005r56172.19.0.75006r6其中salve-master结构是通过下面的redis-trib.rb脚本自动配置所得Redis主从同步Redis集群中的数据库复制是通过主从同步来实现的主节点（Master）把数据发送给从节点（Slave）主从同步的好处在于高可用，Redis节点有冗余设计Redis集群高可用Redis集群中应该包含奇数个Master，至少应该有3个MasterRedis集群中每个Master都应该有SlaveRedisCluster无需配置负载均衡，因为renrenfast后端已经实现了redis负载均衡拉取redis image包括拉取镜像、创建net2网络# yyyyttttwwww的redis已经把配置文件修改完毕 docker pull yyyyttttwwww/redis # 重命名 docker tag yyyyttttwwww/redis redis docker network create --subnet=172.19.0.0/16 net2 docker run -it -d --name r1 -p 5001:6379 --net=net2 --ip=172.19.0.2 redis bash配置Redis节点配置文件路径：/usr/redis/redis.conf重点参数详情daemonize yes # 以后台进程运行cluster-enabled yes # 开启集群cluster-config-file nodes.conf # 集群配置文件cluster-node-timeout 15000 # 超时时间appendonly yes # 开启AOF模式，即日志功能，redis宕机可以根据AOF的日志恢复数据vi /usr/redis/redis.conf # 搜索 不要进入到insert模式，直接输入/命令 /daemonize /cluster-enabled /cluster-config-file /cluster-node-timeout /appendonly # 保存 :wq # 运行redis cd /usr/redis/src ./redis-server ../redis.conf首个节点搭建成功。接下来创建第2-6个redis节点并分别启动redis服务（容器跑起来并未自动启动redis服务）：docker run -it -d --name r2 -p 5002:6379 --net=net2 --ip=172.19.0.3 redis bash docker run -it -d --name r3 -p 5003:6379 --net=net2 --ip=172.19.0.4 redis bash docker run -it -d --name r4 -p 5004:6379 --net=net2 --ip=172.19.0.5 redis bash docker run -it -d --name r5 -p 5005:6379 --net=net2 --ip=172.19.0.6 redis bash docker run -it -d --name r6 -p 5006:6379 --net=net2 --ip=172.19.0.7 redis bash docker exec -it r2 /usr/redis/src/redis-server /usr/redis/redis.conf docker exec -it r3 /usr/redis/src/redis-server /usr/redis/redis.conf docker exec -it r4 /usr/redis/src/redis-server /usr/redis/redis.conf docker exec -it r5 /usr/redis/src/redis-server /usr/redis/redis.conf docker exec -it r6 /usr/redis/src/redis-server /usr/redis/redis.conf安装redis-trib.rbredis-tribe是基于Ruby的Redis集群命令行工具使用的是yyyyttttwwww/redis，Ruby脚本和Ruby环境都已安装# 在容器中（进入到6个redis容器中的任意一个，进行如下操作即可组网） cd /usr/redis mkdir cluster cp /usr/redis/src/redis-trib.rb /usr/redis/cluster/ cd /usr/redis/cluster apt-get install ruby apt-get install rubygems gem install redis创建Redis集群--replicas 1 表示每个主节点创建一个从节点docker exec -it r1 bash # 进到容器中 ./redis-trib.rb create --replicas 1 172.19.0.2:6379 172.19.0.3:6379 172.19.0.4:6379 172.19.0.5:6379 172.19.0.6:6379 172.19.0.7:6379 >>> Creating cluster >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 172.19.0.2:6379 172.19.0.3:6379 172.19.0.4:6379 Adding replica 172.19.0.5:6379 to 172.19.0.2:6379 Adding replica 172.19.0.6:6379 to 172.19.0.3:6379 Adding replica 172.19.0.7:6379 to 172.19.0.4:6379 M: 74734d141f051f70a1b59c883dd1a7368e803ab9 172.19.0.2:6379 slots:0-5460 (5461 slots) master M: 1b49e26dc5277e19bbc987329bad7f1fb4a36bb1 172.19.0.3:6379 slots:5461-10922 (5462 slots) master M: 353faec9b383c10928ca476f361163b7165e9565 172.19.0.4:6379 slots:10923-16383 (5461 slots) master S: 19c4869c9f70a8f64808e399ce6e3b84b834d03e 172.19.0.5:6379 replicates 74734d141f051f70a1b59c883dd1a7368e803ab9 S: aa6fdd39064848a56fff6d81268d0593fa11a588 172.19.0.6:6379 replicates 1b49e26dc5277e19bbc987329bad7f1fb4a36bb1 S: 7218cd5956cb6bd5d3f7f42228a4ec1a5d3ec4a4 172.19.0.7:6379 replicates 353faec9b383c10928ca476f361163b7165e9565 Can I set the above configuration? (type 'yes' to accept):yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join..... >>> Performing Cluster Check (using node 172.19.0.2:6379) M: 74734d141f051f70a1b59c883dd1a7368e803ab9 172.19.0.2:6379 slots:0-5460 (5461 slots) master M: 1b49e26dc5277e19bbc987329bad7f1fb4a36bb1 172.19.0.3:6379 slots:5461-10922 (5462 slots) master M: 353faec9b383c10928ca476f361163b7165e9565 172.19.0.4:6379 slots:10923-16383 (5461 slots) master M: 19c4869c9f70a8f64808e399ce6e3b84b834d03e 172.19.0.5:6379 slots: (0 slots) master replicates 74734d141f051f70a1b59c883dd1a7368e803ab9 M: aa6fdd39064848a56fff6d81268d0593fa11a588 172.19.0.6:6379 slots: (0 slots) master replicates 1b49e26dc5277e19bbc987329bad7f1fb4a36bb1 M: 7218cd5956cb6bd5d3f7f42228a4ec1a5d3ec4a4 172.19.0.7:6379 slots: (0 slots) master replicates 353faec9b383c10928ca476f361163b7165e9565 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered创建成功。访问任意一个节点即可测试。docker exec -it r1 bash /usr/redis/src/redis-cli -c 127.0.0.1:6379>set a 10 -> Redirected to slot [15495] located at 172.19.0.4:6379 # OK 172.19.0.4:6379> get a "10"/usr/redis/src/redis-cli -c 中的 -c 表示连接到redis clusterset a 10时，经过集群计算数据存储到172.19.0.4节点上尝试把172.19.0.4（master）节点挂掉，并查看cluster节点状态：docker container pause r3 docker exec -it r1 bash root@a62d4263df70:/# /usr/redis/src/redis-cli -c 127.0.0.1:6379> cluster nodes 353faec9b383c10928ca476f361163b7165e9565 172.19.0.4:6379 master,fail - 1661308551183 1661308547172 3 connected 1b49e26dc5277e19bbc987329bad7f1fb4a36bb1 172.19.0.3:6379 master - 0 1661308571248 2 connected 5461-10922 7218cd5956cb6bd5d3f7f42228a4ec1a5d3ec4a4 172.19.0.7:6379 master - 0 1661308572250 7 connected 10923-16383 aa6fdd39064848a56fff6d81268d0593fa11a588 172.19.0.6:6379 slave 1b49e26dc5277e19bbc987329bad7f1fb4a36bb1 0 1661308574256 5 connected 74734d141f051f70a1b59c883dd1a7368e803ab9 172.19.0.2:6379 myself,master - 0 0 1 connected 0-5460 19c4869c9f70a8f64808e399ce6e3b84b834d03e 172.19.0.5:6379 slave 74734d141f051f70a1b59c883dd1a7368e803ab9 0 1661308573254 4 connected 127.0.0.1:6379> get a -> Redirected to slot [15495] located at 172.19.0.7:6379 "10"并且此时get a，依然可以使用备用节点获取到数据。恢复r3节点，再去get a，依然可以取到数据，只不过主备节点身份互换。docker container unpause r3 # 再次进入到r1 docker exec -it r1 bash /usr/redis/src/redis-cli -c 127.0.0.1:6379> get a -> Redirected to slot [15495] located at 172.19.0.7:6379 "10" 172.19.0.7:6379> cluster nodes aa6fdd39064848a56fff6d81268d0593fa11a588 172.19.0.6:6379 slave 1b49e26dc5277e19bbc987329bad7f1fb4a36bb1 0 1661308887352 5 connected 1b49e26dc5277e19bbc987329bad7f1fb4a36bb1 172.19.0.3:6379 master - 0 1661308884344 2 connected 5461-10922 353faec9b383c10928ca476f361163b7165e9565 172.19.0.4:6379 slave 7218cd5956cb6bd5d3f7f42228a4ec1a5d3ec4a4 0 1661308889360 7 connected 7218cd5956cb6bd5d3f7f42228a4ec1a5d3ec4a4 172.19.0.7:6379 myself,master - 0 0 7 connected 10923-16383 19c4869c9f70a8f64808e399ce6e3b84b834d03e 172.19.0.5:6379 slave 74734d141f051f70a1b59c883dd1a7368e803ab9 0 1661308888356 4 connected 74734d141f051f70a1b59c883dd1a7368e803ab9 172.19.0.2:6379 master - 0 1661308886350 1 connected 0-5460 172.19.0.7:6379>172.19.0.7变为master，172.19.0.4变为salve。参考资料PXC8.0集群搭建遇见的坑Docker启动、进入容器，查看容器logDocker命令_各种参数简介（run、v、rm、-w、-u、-e）percona-xtrabackup官方文档 - Docker部署方法mysql 启动失败 详情,MySQL 启动失败的常见原因
Docker第一章 Docker的安装和介绍容器技术的介绍容器引领基础架构90年代 PC00年代 虚拟化10年代 cloud11年代 container容器是一种快速打包技术标准化轻量级易移植Linux container容器技术的诞生（2008年）解决了IT世界里“集装箱运输”的问题。Linux container（LXC）是一种内核轻量级的操作系统层虚拟化技术。LXC主要由namespace与Cgroup两大机制来保证实现。namespace资源隔离Cgroup资源管理，如CPU/MEM的限制容器的快速发展与普及2020年，全球>50%公司在生产环境中使用container——Analysis By Gartner容器的标准化docker != container2015年，Google、Docker、Redhat等厂商联合发起OCI（（Open Container Initiative）组织，致力于容器的标准化。容器关乎速度软件开发编译构建测试部署更新故障恢复Linux上安装Dockerhttps://get.docker.com上有安装脚本执行curl -fsSL get.docker.com -o get-docker.sh即可安装完成后sudo systemctl start docker sudo docker version出现client、server的信息即为安装成功。第二章 容器快速上手认识docker命令行非root账户下需要在docker命令前加sudodocker version docker info docker container ps docker container ps -a docker image ls docker image rm nginxdocker的基本操作新旧格式命令均兼容建议学习新格式命令新格式命令旧格式命令docker container run nginxdocker run nginxdocker container stop nginxdocker stop 1a6fcdocker container lsdocker psdocker container ls -adocker ps -adocker container rm 1a6fcdocker rm 1a6fc命令小技巧之批量操作例如存在多个container时：docker container af 87 c2 4c当要操作的容器变多时，上面的命令不再方便，可以如下操作：docker container ps -aq# 批量停止 docker container stop $(docker container ps -aq) # 批量删除 docker container rm $(docker container ps -aq)不能删除一个正在运行的containercontainer的attached和detached模式attached模式（前台运行模式）docker container run -p 80:80 nginx执行的结果直接输出到命令行里，按ctrl+c即可中断。Windows系统上的attached模式并非完整attached模式。detached模式（后台运行模式）docker run --detached ...简写为docker run -d ...docker run -d -p 80:80 nginxdetach进入attach模式docker attach b2容器的交互模式docker container run -d -p 80:80 nginx docker container ps # 查看log docker container logs d02 # 动态查看log docker container -f logs d02交互式运行模式（-it）docker container run -it ubuntu shdetached模式容器进入交互式模式# 执行shell docker exec -it dc3 sh使用busybox镜像熟悉交互模式docker container run -it busybox shWindows是如何运行docker engine的hyper-v manageWindows系统使用基于hyper-v的虚拟机WSL2 based engine也可以使用WSL2替代hyper-v在Windows10中安装WSL2（Ubuntu），在WSL2安装docker容器和虚拟机的区别docker container run -it创建一个容器并进入交互式模式docker container run -it busybox shdocker container exec -it 在一个已经运行的容器里执行一个额外的commanddocker container run -d nginx docker container exec -it 33d sh容器和虚拟机 Container vs VM容器不是Mini虚拟机容器其实是进程containers are just processes容器中的进程被限制了对CPU内存等资源的访问当进程停止后，容器就退出了实验docker container run -d nginx docker container ps # 查看容器内的进程 docker container top fcfNote可以使用pstree命令h直观的看出进程之间的创建附属关系。在Ubuntu20.04中，pstree命令需额外安装，可以使用yum install psmisc或者sudo apt-get install psmisc安装。docker container run背后发生了什么docker container run -d publish 80:80 --name webhost nginx 在本地查找是否有nginx这个image镜像，如果没发现，会执行2、3步去远程的image register查找nginx镜像（默认的register是Docker Hub）下载最新版本的nginx镜像（nginx:latest默认）基于nginx镜像来创建一个新的容器，并准备运行docker engine分配给这个容器一个虚拟IP地址在宿主机上打开80端口并把容器的80端口转发到宿主机的80端口上启动容器，运行指定命令（这里是一个shell脚本去启动nginx）第三章 镜像的创建管理和发布镜像的获取pull from registry（online）从registry拉取public（公有）private（私有）build from Dockerfile（online）从Dockerfile构建load from file（offline）文件导入（离线）镜像的registryDocker HubQuay.io（Redhat）镜像的获取查看和删除docker iamge# 拉取官方nginx（默认最新） docker pull nginx # 查看镜像列表 docker image ls # 拉取指定版本的nginx docker image pull nginx:1.20.0 # 查看镜像列表 docker image ls# 从quay.io拉取镜像 docker image pull quay.io/bitnami/nginx docker image ls# 获取镜像的信息，参数是镜像的ID docker image inspect f0b8a9a54136Architecture架构，如amd64Layers分层，即镜像分层，每一层对应的哈希值# 删除镜像，参数为镜像ID docker image rm 092如果要删除的镜像，有一些正在使用的容器，将无法删除。先要把对应container删掉，才能删除镜像。镜像的导入导出docker image save nginx:1.20.0 -o nginx.image# 导入镜像（离线） docker image load -i ./nginx.imageDockerfile介绍Dockerfile是用于构建docker镜像的文件Dockerfile里包含了构建镜像所需的“指令”Dockerfile有其特定的语法规则举例：执行一个python程序容器即进程，所以镜像就是一个运行这个进程所需要的环境。加入我们要在一台Ubuntu20.04上运行hello.py的python程序hello.py的文件内容print("hello docker")第一步，准备python环境apt update && \ apt install software-properties-common && \ add-apt-repository ppa:deadsnakes/ppa && \ apt install python3.9第二步，运行hello.py$ python3 hello.py hello docker 一个Dockerfile的基本结构FROMRUNADDCMDDockerfileFROM ubuntu:20.04 RUN apt update && \ apt install software-properties-common && \ add-apt-repository ppa:deadsnakes/ppa && \ apt install python3.9 # 将当前目录下的hello.py添加到容器内的根目录下 ADD hello.py / CMD ["python3", "/hello.py"]镜像的构架和分享hello.pyDockerfilehello.py与Dockerfile在同一个目录里# 如果不加版本号，默认为latest docker image build -t hello ./ # 带版本号 docker image build -t hello:1.0 ./之前做过一次构建，二次构建会很快清除本地构建的镜像缓存，再构建docker run -it hello执行之后，进程就停下来了，container也就停止了把镜像push到Docker Hub上，这样任何人都可以拉取如果push到Docker Hub上，镜像名称需要遵循ID/NAME:version的命名规范，比如docker image build -t billaday/hello:1.0 ./但是如果已经构建出来了，仅仅是名称不符合规范，这是根据已经存在的image进行操作# 给image打一个新标签billaday/hello:1.0 docker image tag hello billaday/hello:1.0推送镜像到Docker Hub上# 先登录，输入账户密码，出现Login Succeeded即登陆成功 docker login Username: billaday Password: Login Succeededdocker image push billaday/hello:1.0验证推送的镜像docker pull billaday/hello:1.0 docker image ls docker run -it billaday/hello:1.0通过commit创建镜像docker container run -d -p 80:80 nginx docker container ls # 进入刚刚创建的nginx容器的交互式模式 docker container exec -it 07e sh # 修改index.html文件 cd /usr/share/nginx/html echo " hello docker " > index.html # 提交修改生成新镜像 docker container commit 07e billaday/nginx # 查看新的image docker image ls创建一台Ubuntu21.04的机器，并安装python3.9环境，运行一个hello.py程序docker container run -it ubuntu:21.04 sh # 进入容器shell，使用ppa安装python3.9 apt update && \ apt install software-properties-common && \ add-apt-repository ppa:deadsnakes/ppa && \ apt install python3.9 ls echo "print('hello docker')" python3 hello.py # 退出容器 exit # 查看容器列表 docker container ls -a docker container commit 4a2 billaday/python-demo # 运行新创建的镜像，会直接打印出hello docker docker container run -it billaday/python-demo python3 /hello.pyscratch镜像scratch是一个空的镜像，可以基于他创建一个自己的镜像先将下面的hello.cpp编译成二进制文件hello#include <stdio.h> int main(int argc, char *argv[]) { printf("hello\n"); return 0; }编译时一定要加上--static参数# 使用gcc gcc --satic -o hello hello.cpp # 或g++进行编译 g++ --satic -o hello hello.cpp再在当前目录下新建并编写Dockerfile文件FROM scratch ADD hello / CMD ["/hello"]构建docker imagedocker build -t hello ./查看docker imagedocker image ls运行基于hello镜像的容器docker container run -it hello通过docker image history命令查看分层docker iamge history hello可以看到二进制文件hello的大小与容器的大小相同，均为872KB第四章 Dockerfile完全指南基础镜像的选择官方镜像优于非官方镜像，如果没有官方镜像，则尽量选择Dockerfile开源的固定版本tag而不是每次都是用latest尽量选择体积小的镜像## 通过RUN执行指令可以安装、下载软件配置预置环境以安装ipconfig、git、repo为例Ubuntu21.04上需要执行以下指令apt-get update apt-get insatll net-tools apt-get install git mkdir ~/bin curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo chmod a+x ~/bin/repo echo "export PATH=~/bin:$PATH" >> ~/.bashrc source ~/.bashrcDockerfileFROM ubuntu:21.04 RUN apt-get update RUN apt-get insatll net-tools RUN apt-get install git RUN mkdir ~/bin RUN curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo RUN chmod a+x ~/bin/repo RUN echo "export PATH=~/bin:$PATH" >> ~/.bashrc RUN source ~/.bashrc不推荐频繁执行RUN命令，因为每执行一次RUN，都会产生一层image layer，而且镜像可能打出来体积会更大，建议合并成一个指令改进版DockerfileFROM ubuntu:21.04 RUN apt-get update && \ apt-get insatll net-tools -y && \ apt-get install git -y && \ mkdir ~/bin && \ curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo && \ chmod a+x ~/bin/repo && \ echo "export PATH=~/bin:$PATH" >> ~/.bashrc && \ source ~/.bashrc文件复制和目录操作文件复制有两种方式，COPY和ADDCOPY和ADD都可以把local的一个文件复制到镜像里，如果目标目录不存在，则会自动创建复制普通文件FROM python:3.9.5-alpine3.13 COPY hello.py /app/hello.py比如把local的hello.py复制到容器的/app目录下，如果app目录不存在，则会自动创建复制压缩文件ADD比COPY高级一点的地方就是，如果复制的是一个gzip等压缩文件，ADD会帮助我们自动解压缩文件。FROM python:3.9.5-alpine3.13 ADD hello.tar.gz /app/目录切换WORKDIR，当目录不存在时会自动创建FROM python:3.9.5-alpine3.13 WORKDIR /app COPY hello.py hello.py构建参数和环境变量(ARG vs ENV)使用ARG的Dockerfile使用ENV的Dockerfile使用arg创建的变量，变量只能存在于构建当中，不会保存在image中使用env创建的变量，不仅在build中可以使用，也可以作为image的环境变量，在容器中可获取区别图示：# 通过--build-arg动态指定arg的VERSION=2.0.0 docker iamge build -f ./Dockerfile-arg -t ipinfo-arg-2.0.0 --build-arg VERSION=2.0.0上面这个是arg比env方便的地方CMD容器启动命令CMD可以用来设置容器启动时默认执行的命令容器启动时默认执行的命令如果docker container run自动容器时指定了其他命令，则CMD会被忽略如果定义多个CMD，只有最后一个会执行例如 如下Dockerfile# 把已经退出的容器全部删除 docker system prune -f # 把没有使用的镜像全部删除 docker image prune -a # build镜像 docker image build -t ipinfo . # 创建容器 docker container run -it ipinfo我们会发现run之后容器会自动进入shell，但我们Dockerfile没有设置CMD。这是因为Ubuntu基础镜像里有CMD使用--rm，容器运行结束后会自动删除docker container run --rm -it ipinfo ipinfo 8.8.8.8容器启动命令ENTRYPOINTENTRYPOINT也可以设置容器启动时要执行的命令，但是和CMD是由去别的。CMD设置的命令，可以在docker container run时传入其他命令，覆盖掉CMD命令，但是ENTERPOINT所设置的命令是一定会被执行的。ENTRYPOINT和CMD可以联合使用，ENTRYPOINT设置命令，CMD传递参数实验Dockerfile-cmdFROM ubuntu:21.04 CMD ["echo", "hello docker"]Dockerfile-entrypointFROM ubuntu:21.04 ENTRYPOINT ["echo", "hello docker"]DockerfileFROM ubuntu:21.04 ENTRYPOINT ["echo"] CMD []Shell格式和Exec格式CMD和ENTRYPOINT同时支持shell格式和exec格式。Shell格式CMD echo "hello docker"ENTRYPOINT echo "hello docker"Exec格式CMD ["echo", "hello docker"]ENTRYPOINT ["echo", "hello docker"]注意shell脚本的问题FROM ubuntu:21.04 ENV NAME=docker CMD echo "hello $NAME"上面的写法是可以的，但是如果把shell写法改成Exec格式，下面的写法是不行的：FROM ubuntu:21.04 ENV NAME=docker CMD ["echo", "hello $NAME"]这样会打印出hello $NAME，而不是hello docker，需要以shell脚本的方式去执行FROM ubuntu:21.04 ENV NAME=docker CMD ["sh", "-c", "echo hello $NAME"]构建一个Python Flask镜像from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): return 'hello, world!'# 如果没安装虚拟环境，先安装vnev pip3 install --user virtualenv # 在当前目录下创建python虚拟环境 python3 -m venv env # 使虚拟环境生效 ./env/bin/activate # 安装flask pip3 install flask # 设置flask环境变量 export FLASK_APP=app.py # 运行app.py flask run本地运行成功，下面编写DockerfileFROM python:3.9.5-slim # 不要写/src，要写/src/，不然会识别成文件 # COPY app.py /src × # COPY app.py /src/ √ # 推荐 COPY app.py /src/app.py RUN pip install flask WORKDIR /src ENV FLASK_APP=app.py # 暴露5000端口 EXPOSE 5000 CMD ["flask", "run", "-h", "0.0.0.0"]构建镜像docker image build -t flask-demo ./测试运行docker container run -d -p 5000:5000 flask-demo部署成功！Dockerfile最佳实践合理使用缓存这里使用了缓存，可以加速构建一旦某一层命令发生变化，则从这条命令开始往后构建docker镜像都不使用缓存制作镜像时，应当把固定的、不经常变化的，放在Docekrfile的前面，从而使用缓存加速构架dockerignoreDockerfile构建image的路径，使用.dockerignore忽略掉镜像不需要的文件（夹），例如下图所示，其中.vscode、env均不需要打包到镜像中，如果没有忽略，镜像体积会冗余。.dockerignore也可以保护本地的数据不传输到server、不包含到镜像中。创建.dockerignore，其使用方法与.gitignore基本相同.vscode/ env/再次打包后，会发现image镜像体积减少多阶段构建hello.c#include <stdio.h> int main(int argc, char *argv[]) { printf("hello\n"); return 0; }DockerfileFROM gcc:9.4 COPY hello.c /src/hello.c WORKDIR /src RUN gcc --static -o hello hello.c ENTRYPOINT ["/src/hello"] CMD []build imagedocker build -t hello-gcc -f Dockerfile .镜像有1.14GB，很大，基础镜像gcc占用了很大空间。实际需求是仅仅需要运行编译好的hello二进制文件，这里选择gcc镜像，是因为要用到gcc进行编译，但是编译之后，就不需要gcc了。这时就要引入docker的多阶段构建先使用gcc基础镜像，对hello.c进行编译在选用alpine:3.13.5基础镜像运行hello二进制文件新的Dockerfile文件FROM gcc:9.4 AS builder COPY hello.c /src/hello.c WORKDIR /src RUN gcc --static -o hello hello.c # =============================== FROM alpine:3.13.5 COPY --from=builder /src/hello /src/hello ENTRYPOINT [ "/src/hello" ] CMD []最后构建的镜像大小只有6.55MB尽量使用非root用户原文链接root的危险性docker的root权限一直是其遭受诟病的地方，docker的root权限有那么危险么？我们举个例子。假如我们有一个用户，叫demo，它本身不具有sudo的权限，所以就有很多文件无法进行读写操作，比如/root目录它是无法查看的。[demo@docker-host ~]$ sudo ls /root [sudo] password for demo: demo is not in the sudoers file. This incident will be reported. [demo@docker-host ~]$但是这个用户有执行docker的权限，也就是它在docker这个group里。[demo@docker-host ~]$ groups demo docker [demo@docker-host ~]$ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE busybox latest a9d583973f65 2 days ago 1.23MB [demo@docker-host ~]$这时，我们就可以通过Docker做很多越权的事情了，比如，我们可以把这个无法查看的/root目录映射到docker container里，你就可以自由进行查看了。[demo@docker-host vagrant]$ docker run -it -v /root/:/root/tmp busybox sh / # cd /root/tmp ~/tmp # ls anaconda-ks.cfg original-ks.cfg ~/tmp # ls -l total 16 -rw------- 1 root root 5570 Apr 30 2020 anaconda-ks.cfg -rw------- 1 root root 5300 Apr 30 2020 original-ks.cfg ~/tmp #更甚至我们可以给我们自己加sudo权限。我们现在没有sudo权限[demo@docker-host ~]$ sudo vim /etc/sudoers [sudo] password for demo: demo is not in the sudoers file. This incident will be reported. [demo@docker-host ~]$但是我可以给自己添加。[demo@docker-host ~]$ docker run -it -v /etc/sudoers:/root/sudoers busybox sh / # echo "demo ALL=(ALL) ALL" >> /root/sudoers / # more /root/sudoers | grep demo demo ALL=(ALL) ALL然后退出container，bingo，我们有sudo权限了。[demo@docker-host ~]$ sudo more /etc/sudoers | grep demo demo ALL=(ALL) ALL [demo@docker-host ~]$如何使用非root用户以运行一个flask-hello-world项目为例准备两个Dockerfile文件，前者不指定用户运行flask（即默认是root用户运行），后者指定flask用户（用户组）运行flask第一个Dockerfile如下FROM python:3.9.5-slim RUN pip install flask COPY app.py /src/app.py WORKDIR /src ENV FLASK_APP=app.py EXPOSE 5000 CMD ["flask", "run", "-h", "0.0.0.0"]假设构建的镜像名称为flask-demo第二个Dockerfile，使用非root用户构建镜像，镜像名称为flask-no-root通过groupadd和useradd创建一个flask用户及用户组通过USER指定后面的命令要以flask用户运行FROM python:3.9.5-slim RUN pip install flask && \ groupadd -r flask && useradd -r -g flask flask && \ mkdir /src && \ chown -R flask:flask /src USER flask COPY app.py /src/app.py WORKDIR /src ENV FLASK_APP=app.py EXPOSE 5000 CMD ["flask", "run", "-h", "0.0.0.0"]第五章 Docker的存储默认情况下，在运行中的容器里创建的文件，被保存在一个可写的容器层：如果容器被删除了，则数据也没有了这个可写的容器是和特定的容器绑定的，也就是这些数据无法方便的和其他容器共享Docker主要提供了两种方式做数据的持久化Data Volume，由Docker管理，（/var/lib/volumes/Linux），持久化数据的最好方式Bind Mount，由用户指定存储的数据具体mount在系统什么位置Data Volume示例：通过Docker container执行计划任务环境准备DockerfileFROM alpine:latest RUN apk update RUN apk --no-cache add curl ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.1.12/supercronic-linux-amd64 \ SUPERCRONIC=supercronic-linux-amd64 \ SUPERCRONIC_SHA1SUM=048b95b48b708983effb2e5c935a1ef8483d9e3e RUN curl -fsSLO "$SUPERCRONIC_URL" \ && echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c - \ && chmod +x "$SUPERCRONIC" \ && mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \ && ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic COPY my-cron /app/my-cron WORKDIR /app # RUN cron job CMD ["/usr/local/bin/supercronic", "/app/my-cron"]my-cron脚本*/1 * * * * date >> /app/test.txt构建镜像docker image build -t my-cron .Could not resolve host xxx 错误解决在构建镜像的时候，可能会报如下错误：Could not resolve host github.com，这是因为docker容器没有配置DNS服务器的，导致域名无法解析的缘故，我们通过查看宿主机的DNS服务器，调整docker的DNS配置相同即可。首先在宿主机执行nmcli dev show | grep 'IP4.DNS'出现IP4.DNS[1]: 192.168.4.146 IP4.DNS[2]: 192.168.4.147记住上面查询到的DNS服务器地址，编辑/etc/docker/daemon.json文件，在里面添加DNS地址（没有daemon.json创建）## 此处的dns是填写你前面查询出来的 ## 有几个写几个 { "dns": ["192.168.4.146", "192.168.4.147"] } #示例文件如下 { "registry-mirrors": ["https://nrbewqda.mirror.aliyuncs.com"], "dns": ["208.116.3.201"] }而后重启docker，再编译即可service docker restart编译成功运行容器并进入到容器中docker container run -d my-cron docker container ls docker container exec -it 087 sh容器内/app # ls my-cron test.txt /app # cat test.txt Tue May 10 06:36:00 UTC 2022 Tue May 10 06:37:00 UTC 2022停止容器后再启动容器，进入到容器内数据仍然在root@bill-lixiang-a:~# docker container stop 087 087 root@bill-lixiang-a:~# docker container start 087 087 root@bill-lixiang-a:~# docker container exec -it 087 sh /app # ls my-cron test.txt /app # more test.txt Tue May 10 06:36:00 UTC 2022 Tue May 10 06:37:00 UTC 2022 Tue May 10 06:38:00 UTC 2022 Tue May 10 06:39:00 UTC 2022但是删除容器后，数据就会丢失。如何把数据持久化的存到磁盘上，使用DataVolume实现数据持久化。在Dockerfile内添加VOLUME ["/app"]语句FROM alpine:latest RUN apk update RUN apk --no-cache add curl ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.1.12/supercronic-linux-amd64 \ SUPERCRONIC=supercronic-linux-amd64 \ SUPERCRONIC_SHA1SUM=048b95b48b708983effb2e5c935a1ef8483d9e3e RUN curl -fsSLO "$SUPERCRONIC_URL" \ && echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c - \ && chmod +x "$SUPERCRONIC" \ && mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \ && ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic COPY my-cron /app/my-cron WORKDIR /app VOLUME ["/app"] # RUN cron job CMD ["/usr/local/bin/supercronic", "/app/my-cron"]重新构建image镜像并运行容器docker image build -t my-cron . docker container run -d my-cron docker container ls docker container exec -it 792 sh # ============================= /app # ls my-cron test.txt /app # more test.txt Tue May 10 06:36:00 UTC 2022查看volumedocker volume ls # 查看某个volume详细信息 docker volume inspect f9d264e01253a153c16f3701cff3c7fe6c27c857489a99e83a7a12d834e41659 [ { "CreatedAt": "2022-05-10T14:36:00+08:00", "Driver": "local", "Labels": null, "Mountpoint": "/var/lib/docker/volumes/f9d264e01253a153c16f3701cff3c7fe6c27c857489a99e83a7a12d834e41659/_data", "Name": "f9d264e01253a153c16f3701cff3c7fe6c27c857489a99e83a7a12d834e41659", "Options": null, "Scope": "local" } ]查看挂载点文件夹内容$ ls /var/lib/docker/volumes/f9d264e01253a153c16f3701cff3c7fe6c27c857489a99e83a7a12d834e41659/_data my-cron test.txt​ 可以看到挂载点在物理机上面存储了，实现了数据的持久化。此时即使删除container，volume也不会被删除。但是如果删除了容器，再次创建个新的容器，也会重新创建一个新的volume。而在实际的场景中，往往我们需要使用原来的volume内的数据，这样虽然实现了数据持久化，但还不是很方便使用。​ 我们可以给volume起个名字，每次创建新的容器，使用相同的volume名字即可。# 删除没有使用的volume $ docker volume prune # 创建容器的时候给volume起个名 $ docker container run -d -v cron-data:/app my-cron # 可以看到volume列表有一个cron-data的卷 $ docker volume ls DRIVER VOLUME NAME local cron-data # 查看volume详细信息 $ docker volume inspect cron-data [ { "CreatedAt": "2022-05-10T14:57:59+08:00", "Driver": "local", "Labels": null, "Mountpoint": "/var/lib/docker/volumes/cron-data/_data", "Name": "cron-data", "Options": null, "Scope": "local" } ] # 删除容器，可以看到volume数据还在 $ docker container rm -f 43a $ more /var/lib/docker/volumes/cron-data/_data/test.txt Tue May 10 06:58:00 UTC 2022 Tue May 10 06:59:00 UTC 2022 Tue May 10 07:00:00 UTC 2022此时我们可以再去创建一个container，并挂载先前的volume$ docker container run -d -v cron-data:/app my-cron $ docker container exec -it f79 sh /app # more test.txt Tue May 10 06:58:00 UTC 2022 Tue May 10 06:59:00 UTC 2022 Tue May 10 07:00:00 UTC 2022 Tue May 10 07:05:00 UTC 2022可以看到先前volume的数据被挂载上来，后面新的log会在之前log的基础上追加。注：只能在Linux上直接查看volume的数据，Windows上也可以查看，但是比较麻烦，可以网上搜索。上面实验的命令均在Linux环境下进行。Data Volume 练习MySQL准备镜像$ docker pull mysql:5.7 $ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE mysql 5.7 8aa4b5ffb001 12 days ago 462MB创建容器$ docker container run --name some-mysql -e MYSQL_ROOT_PASSWORD=123456 -d -v mysql-data:/var/lib/mysql mysql:5.7 ae04887aa2787ed9663fc54bf0ebca131ab71d39df4790b4bf0fe1ec0381ea6c $ docker container ls CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ae04887aa278 mysql:5.7 "docker-entrypoint.s…" 20 seconds ago Up 19 seconds 3306/tcp, 33060/tcp some-mysql $ docker volume inspect mysql-data [ { "CreatedAt": "2022-05-10T15:37:28+08:00", "Driver": "local", "Labels": null, "Mountpoint": "/var/lib/docker/volumes/mysql-data/_data", "Name": "mysql-data", "Options": null, "Scope": "local" } ] # 进入到MySQL容器 $ docker container exec -it ae0 sh数据库写入数据MySQL容器mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.38 MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 4 rows in set (0.00 sec) mysql> create database demo; Query OK, 1 row affected (0.00 sec) mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | demo | | mysql | | performance_schema | | sys | +--------------------+ 5 rows in set (0.00 sec) mysql> exit Bye退出MySQL容器exit查看mysql-data volume内的数据$ ls /var/lib/docker/volumes/mysql-data/_data auto.cnf demo ibtmp1 server-cert.pem ca-key.pem ib_buffer_pool mysql server-key.pem ca.pem ibdata1 performance_schema sys client-cert.pem ib_logfile0 private_key.pem client-key.pem ib_logfile1 public_key.pem重新创建容器删除MySQL容器，删除container并不会删除volumedocker container rm -f ae0创建新的MySQL容器并挂载先前的volume，进入容器，查看database$ docker container run --name some-mysql -e MYSQL_ROOT_PASSWORD=123456 -d -v mysql-data:/var/lib/mysql mysql:5.7 6a2e3ae691052e19d6cd8169496f107574a7f6ce1c1a32a5066a5cf03772c06e $ docker container exec -it 6a2 shMySQL容器mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.38 MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | demo | | mysql | | performance_schema | | sys | +--------------------+ 5 rows in set (0.00 sec) mysql>可以看到之前的demo数据库已经在新的MySQL容器中了。数据持久化之Bind Mount还是使用之前my-cron的Dockerfile不在Dockerfile文件里指定VOLUME ["/app"]FROM alpine:latest RUN apk update RUN apk --no-cache add curl ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.1.12/supercronic-linux-amd64 \ SUPERCRONIC=supercronic-linux-amd64 \ SUPERCRONIC_SHA1SUM=048b95b48b708983effb2e5c935a1ef8483d9e3e RUN curl -fsSLO "$SUPERCRONIC_URL" \ && echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c - \ && chmod +x "$SUPERCRONIC" \ && mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \ && ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic COPY my-cron /app/my-cron WORKDIR /app # RUN cron job CMD ["/usr/local/bin/supercronic", "/app/my-cron"]编译镜像docker image build -t my-cron .​ 下面由于我们将/root/app目录映射到容器里的/app目录，需要把my-cron脚本也复制到物理机的/root/app目录下，否则运行会提示open /app/my-cron: no such file or directory使用Bind Mount运行容器cp /home/bill/data/studyDocker/mycron/my-cron /root/app/ # 直接绑定路径即可(-v /root/app:/app) $ docker container run -d -v /root/app:/app my-cron $ ls /root/app my-cron test.txt $ more /root/app/test.txt Wed May 11 01:36:00 UTC 2022​ 可以看到容器中的运行结果输出到了物理机上。Bind Mount练习 - Docker开发环境场景：本地需要C语言的开发环境来编译C语言的代码，但是本地没有C语言开发环境。此时可以使用Docker的Bind Mount待编译调试文件hello.c文件路径在/home/bill/data/studyDocker/mycppvoid main(int argc, char *argv[]) { printf("hello %s\n", argv[argc - 1]); }拉取gcc9.4镜像，运行容器docker image pull gcc:9.4 # 这样运行，默认执行命令为bash，运行一下会退出，需要用交互式运行容器 docker container run -d -v /home/bill/data/studyDocker/mycpp:/root gcc:9.4 # -it运行容器 docker container run -it -v /home/bill/data/studyDocker/mycpp:/root gcc:9.4进入到容器root@1bd22d6de49f:/# cd ~ root@1bd22d6de49f:~# ls hello.c root@1bd22d6de49f:~# gcc -o hello hello.c hello.c: In function 'main': hello.c:2:5: warning: implicit declaration of function 'printf' [-Wimplicit-function-declaration] 2 | printf("hello %s\n", argv[argc - 1]); | ^~~~~~ hello.c:2:5: warning: incompatible implicit declaration of built-in function 'printf' hello.c:1:1: note: include '<stdio.h>' or provide a declaration of 'printf' +++ |+#include <stdio.h> 1 | void main(int argc, char *argv[]) { root@1bd22d6de49f:~# ls hello hello.c root@1bd22d6de49f:~# ./hello docker hello docker多个机器之间的容器共享数据Docker的volume支持多种driver。默认创建的volume driver都是local本节我们使用sshfs(ssh file system)的driver，如何让docker使用不在同一台机器上的文件系统做volume环境准备准备三台Linux机器，之间可以通过SSH通信。hostnameipssh usernamessh passworddocker-host1192.168.200.10vagrantvagrantdocker-host2192.168.200.11vagrantvagrantdocker-host3192.168.200.12vagrantvagrant安装sshfs-plugin在其中两台机器上安装一个plugin vieux/sshfsdocker plugin install --grant-all-permissions vieux/sshfs创建volume(名字为sshvolume)docker volume create --driver vieux/sshfs \ -o sshcmd=vagrant@192.168.200.12:/home/vagrant \ -o password=vagrant \ sshvolume### 查看volume$ docker volume ls DRIVER VOLUME NAME vieux/sshfs:latest sshvolume $ docker volume inspect sshvolume [ { "CreatedAt": "0001-01-01T00:00:00Z", "Driver": "vieux/sshfs:latest", "Labels": {}, "Mountpoint": "/mnt/volumes/f59e848643f73d73a21b881486d55b33", "Name": "sshvolume", "Options": { "password": "vagrant", "sshcmd": "vagrant@192.168.200.12:/home/vagrant" }, "Scope": "local" } ]创建容器挂载volumedocker run -it -v sshvolume:/app busybox sh Unable to find image 'busybox:latest' locally latest: Pulling from library/busybox b71f96345d44: Pull complete Digest: sha256:930490f97e5b921535c153e0e7110d251134cc4b72bbb8133c6a5065cc68580d Status: Downloaded newer image for busybox:latest / # / # ls app bin dev etc home proc root sys tmp usr var / # cd /app /app # ls /app # echo "this is ssh volume"> test.txt /app # ls test.txt /app # more test.txt this is ssh volume /app # /app #test.txt文件我们可以在docker-host3上看到第六章 Docker的网络网络基础知识回顾当我们访问一个WEB站点时，会触发以下流程：https://www.homenethowto.com/advanced-topics/traffic-example-the-full-picture/网络常用命令IP地址的查看WindowsipconfigLinuxifconfig或ip addr网络连通性测试ping测试IP是否可达ping 127.0.0.1telnet测试端口的连通性telnet www.baidu.com 80traceroute路径探测跟踪Linux使用tracepathtracepath www.baidu.comWindows使用tracerttracert www.baidu.comcurl请求WEB服务http://www.ruanyifeng.com/blog/2019/09/curl-reference.htmlDocker bridge 网络bill@bill-lixiang-a:~$ docker network ls NETWORK ID NAME DRIVER SCOPE 774b5e0089ac bridge bridge local f6fe9fa807b4 host host local 165075680a9c net1 bridge local 1d0b2dcf23fc net2 bridge local fb02fb964d7a none null local bill@bill-lixiang-a:~$ docker network inspect bridge [ { "Name": "bridge", "Id": "774b5e0089ac3c9726ffa5c2a7cb33ce07294d325a24ddcedb6dacdf0724c172", "Created": "2022-08-18T09:20:17.847442277+08:00", "Scope": "local", "Driver": "bridge", "EnableIPv6": false, "IPAM": { "Driver": "default", "Options": null, "Config": [ { "Subnet": "172.17.0.0/16", "Gateway": "172.17.0.1" } ] }, "Internal": false, "Attachable": false, "Ingress": false, "ConfigFrom": { "Network": "" }, "ConfigOnly": false, "Containers": {}, "Options": { "com.docker.network.bridge.default_bridge": "true", "com.docker.network.bridge.enable_icc": "true", "com.docker.network.bridge.enable_ip_masquerade": "true", "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0", "com.docker.network.bridge.name": "docker0", "com.docker.network.driver.mtu": "1500" }, "Labels": {} } ]docker0: bridgebrctl 使用前需要安装, 对于CentOS, 可以通过 sudo yum install -y bridge-utils 安装. 对于Ubuntu, 可以通过 sudo apt-get install -y bridge-utilsbill@bill-lixiang-a:~$ brctl show bridge name bridge id STP enabled interfaces br-165075680a9c 8000.02422a4ad8c0 no vetha63ea58 vethccd3776 br-1d0b2dcf23fc 8000.024209dfbea4 no veth38d5113 veth73c221d vethb57f9a8 vethe7eb813 vetheea0225 vetheef23fc docker0 8000.024236471856 no容器对外通信查看路由表bill@bill-lixiang-a:~$ ip route default via 10.248.195.254 dev enp3s0 proto dhcp metric 100 10.248.192.0/22 dev enp3s0 proto kernel scope link src 10.248.192.137 metric 100 169.254.0.0/16 dev enp3s0 scope link metric 1000 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 172.18.0.0/24 dev br-165075680a9c proto kernel scope link src 172.18.0.1 172.19.0.0/16 dev br-1d0b2dcf23fc proto kernel scope link src 172.19.0.1iptables转发规则bill@bill-lixiang-a:~$ sudo iptables --list -t nat Chain PREROUTING (policy ACCEPT) target prot opt source destination DOCKER all -- anywhere anywhere ADDRTYPE match dst-type LOCAL Chain INPUT (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination DOCKER all -- anywhere !localhost/8 ADDRTYPE match dst-type LOCAL Chain POSTROUTING (policy ACCEPT) target prot opt source destination MASQUERADE all -- 172.19.0.0/16 anywhere MASQUERADE all -- 172.18.0.0/24 anywhere MASQUERADE all -- 172.17.0.0/16 anywhere MASQUERADE tcp -- 172.18.0.7 172.18.0.7 tcp dpt:8888 MASQUERADE tcp -- 172.18.0.7 172.18.0.7 tcp dpt:mysql MASQUERADE tcp -- 172.18.0.2 172.18.0.2 tcp dpt:mysql MASQUERADE tcp -- 172.19.0.2 172.19.0.2 tcp dpt:6379 MASQUERADE tcp -- 172.19.0.3 172.19.0.3 tcp dpt:6379 MASQUERADE tcp -- 172.19.0.4 172.19.0.4 tcp dpt:6379 MASQUERADE tcp -- 172.19.0.5 172.19.0.5 tcp dpt:6379 MASQUERADE tcp -- 172.19.0.6 172.19.0.6 tcp dpt:6379 MASQUERADE tcp -- 172.19.0.7 172.19.0.7 tcp dpt:6379 Chain DOCKER (2 references) target prot opt source destination RETURN all -- anywhere anywhere RETURN all -- anywhere anywhere RETURN all -- anywhere anywhere DNAT tcp -- anywhere anywhere tcp dpt:4001 to:172.18.0.7:8888 DNAT tcp -- anywhere anywhere tcp dpt:4002 to:172.18.0.7:3306 DNAT tcp -- anywhere anywhere tcp dpt:mysql to:172.18.0.2:3306 DNAT tcp -- anywhere anywhere tcp dpt:5001 to:172.19.0.2:6379 DNAT tcp -- anywhere anywhere tcp dpt:5002 to:172.19.0.3:6379 DNAT tcp -- anywhere anywhere tcp dpt:5003 to:172.19.0.4:6379 DNAT tcp -- anywhere anywhere tcp dpt:5004 to:172.19.0.5:6379 DNAT tcp -- anywhere anywhere tcp dpt:5005 to:172.19.0.6:6379 DNAT tcp -- anywhere anywhere tcp dpt:5006 to:172.19.0.7:6379更多资料：s1-firewall-ipt-fwd.htmlNAT技术创建和使用自定义Bridgebill@bill-lixiang-a:~$ docker network create -d bridge mybridge 54415915c9f9277d4bf5114f0b1e055f72002cd0d49c80b4f16a505344721cd3 bill@bill-lixiang-a:~$ docker network ls NETWORK ID NAME DRIVER SCOPE 774b5e0089ac bridge bridge local f6fe9fa807b4 host host local 54415915c9f9 mybridge bridge local 165075680a9c net1 bridge local 1d0b2dcf23fc net2 bridge local fb02fb964d7a none null local-d指定driver为bridgebill@bill-lixiang-a:~$ docker network inspect mybridge [ { "Name": "mybridge", "Id": "54415915c9f9277d4bf5114f0b1e055f72002cd0d49c80b4f16a505344721cd3", "Created": "2022-08-24T19:25:54.947586129+08:00", "Scope": "local", "Driver": "bridge", "EnableIPv6": false, "IPAM": { "Driver": "default", "Options": {}, "Config": [ { "Subnet": "172.20.0.0/16", "Gateway": "172.20.0.1" } ] }, "Internal": false, "Attachable": false, "Ingress": false, "ConfigFrom": { "Network": "" }, "ConfigOnly": false, "Containers": {}, "Options": {}, "Labels": {} } ]创建容器并加入到mybridge网络docker container run -d --rm --name box1 --network=mybridge busybox /bin/sh -c "while true; do sleep 3600; done"inspect box1bill@bill-lixiang-a:~$ docker container inspect box1 [ { "Id": "ed3d8c95d32ed3ce29abcced597a4ad5225bdfbb92d9223ec371ea7935322450", "Created": "2022-08-24T11:30:54.203225644Z", "Path": "/bin/sh", "Args": [ "-c", "while true; do sleep 3600; done" ], "State": { "Status": "running", "Running": true, "Paused": false, "Restarting": false, "OOMKilled": false, "Dead": false, "Pid": 909008, "ExitCode": 0, "Error": "", "StartedAt": "2022-08-24T11:30:54.546915117Z", "FinishedAt": "0001-01-01T00:00:00Z" }, "Image": "sha256:7a80323521ccd4c2b4b423fa6e38e5cea156600f40cd855e464cc52a321a24dd", "ResolvConfPath": "/var/snap/docker/common/var-lib-docker/containers/ed3d8c95d32ed3ce29abcced597a4ad5225bdfbb92d9223ec371ea7935322450/resolv.conf", "HostnamePath": "/var/snap/docker/common/var-lib-docker/containers/ed3d8c95d32ed3ce29abcced597a4ad5225bdfbb92d9223ec371ea7935322450/hostname", "HostsPath": "/var/snap/docker/common/var-lib-docker/containers/ed3d8c95d32ed3ce29abcced597a4ad5225bdfbb92d9223ec371ea7935322450/hosts", "LogPath": "/var/snap/docker/common/var-lib-docker/containers/ed3d8c95d32ed3ce29abcced597a4ad5225bdfbb92d9223ec371ea7935322450/ed3d8c95d32ed3ce29abcced597a4ad5225bdfbb92d9223ec371ea7935322450-json.log", "Name": "/box1", "RestartCount": 0, "Driver": "overlay2", "Platform": "linux", "MountLabel": "", "ProcessLabel": "", "AppArmorProfile": "docker-default", "ExecIDs": null, "HostConfig": { "Binds": null, "ContainerIDFile": "", "LogConfig": { "Type": "json-file", "Config": {} }, "NetworkMode": "mybridge", "PortBindings": {}, "RestartPolicy": { "Name": "no", "MaximumRetryCount": 0 }, "AutoRemove": true, "VolumeDriver": "", "VolumesFrom": null, "CapAdd": null, "CapDrop": null, "CgroupnsMode": "host", "Dns": [], "DnsOptions": [], "DnsSearch": [], "ExtraHosts": null, "GroupAdd": null, "IpcMode": "private", "Cgroup": "", "Links": null, "OomScoreAdj": 0, "PidMode": "", "Privileged": false, "PublishAllPorts": false, "ReadonlyRootfs": false, "SecurityOpt": null, "UTSMode": "", "UsernsMode": "", "ShmSize": 67108864, "Runtime": "runc", "ConsoleSize": [ 0, 0 ], "Isolation": "", "CpuShares": 0, "Memory": 0, "NanoCpus": 0, "CgroupParent": "", "BlkioWeight": 0, "BlkioWeightDevice": [], "BlkioDeviceReadBps": null, "BlkioDeviceWriteBps": null, "BlkioDeviceReadIOps": null, "BlkioDeviceWriteIOps": null, "CpuPeriod": 0, "CpuQuota": 0, "CpuRealtimePeriod": 0, "CpuRealtimeRuntime": 0, "CpusetCpus": "", "CpusetMems": "", "Devices": [], "DeviceCgroupRules": null, "DeviceRequests": null, "KernelMemory": 0, "KernelMemoryTCP": 0, "MemoryReservation": 0, "MemorySwap": 0, "MemorySwappiness": null, "OomKillDisable": false, "PidsLimit": null, "Ulimits": null, "CpuCount": 0, "CpuPercent": 0, "IOMaximumIOps": 0, "IOMaximumBandwidth": 0, "MaskedPaths": [ "/proc/asound", "/proc/acpi", "/proc/kcore", "/proc/keys", "/proc/latency_stats", "/proc/timer_list", "/proc/timer_stats", "/proc/sched_debug", "/proc/scsi", "/sys/firmware" ], "ReadonlyPaths": [ "/proc/bus", "/proc/fs", "/proc/irq", "/proc/sys", "/proc/sysrq-trigger" ] }, "GraphDriver": { "Data": { "LowerDir": "/var/snap/docker/common/var-lib-docker/overlay2/f5d046130a18e596a5b80686f82e792152180ef3c7e85ee398df5d15961f8aa0-init/diff:/var/snap/docker/common/var-lib-docker/overlay2/0c139c1fe59d74c09444d322839d9428d88fa2f7f36f17d57afff5ae9f4f56d5/diff", "MergedDir": "/var/snap/docker/common/var-lib-docker/overlay2/f5d046130a18e596a5b80686f82e792152180ef3c7e85ee398df5d15961f8aa0/merged", "UpperDir": "/var/snap/docker/common/var-lib-docker/overlay2/f5d046130a18e596a5b80686f82e792152180ef3c7e85ee398df5d15961f8aa0/diff", "WorkDir": "/var/snap/docker/common/var-lib-docker/overlay2/f5d046130a18e596a5b80686f82e792152180ef3c7e85ee398df5d15961f8aa0/work" }, "Name": "overlay2" }, "Mounts": [], "Config": { "Hostname": "ed3d8c95d32e", "Domainname": "", "User": "", "AttachStdin": false, "AttachStdout": false, "AttachStderr": false, "Tty": false, "OpenStdin": false, "StdinOnce": false, "Env": [ "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" ], "Cmd": [ "/bin/sh", "-c", "while true; do sleep 3600; done" ], "Image": "busybox", "Volumes": null, "WorkingDir": "", "Entrypoint": null, "OnBuild": null, "Labels": {} }, "NetworkSettings": { "Bridge": "", "SandboxID": "b0b56b6232d4a82240f77d8af17da76f9ca5cf827bcbe2cd558c48de10ac6a64", "HairpinMode": false, "LinkLocalIPv6Address": "", "LinkLocalIPv6PrefixLen": 0, "Ports": {}, "SandboxKey": "/run/snap.docker/netns/b0b56b6232d4", "SecondaryIPAddresses": null, "SecondaryIPv6Addresses": null, "EndpointID": "", "Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "IPAddress": "", "IPPrefixLen": 0, "IPv6Gateway": "", "MacAddress": "", "Networks": { "mybridge": { "IPAMConfig": null, "Links": null, "Aliases": [ "ed3d8c95d32e" ], "NetworkID": "54415915c9f9277d4bf5114f0b1e055f72002cd0d49c80b4f16a505344721cd3", "EndpointID": "bdeee9ee6a0c4d80a963975e8e33e95e52de2b22580bc13581990cc36867e463", "Gateway": "172.20.0.1", "IPAddress": "172.20.0.2", "IPPrefixLen": 16, "IPv6Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "MacAddress": "02:42:ac:14:00:02", "DriverOpts": null } } } } ]让box1容器同时也连接bridge网络docker network connect bridge box1inspect box1的network部分"Networks": { "bridge": { "IPAMConfig": {}, "Links": null, "Aliases": [], "NetworkID": "774b5e0089ac3c9726ffa5c2a7cb33ce07294d325a24ddcedb6dacdf0724c172", "EndpointID": "cf6d2b1a8cc6d62b1f19605f73399ae43b37a867039f688510ca8badb312c179", "Gateway": "172.17.0.1", "IPAddress": "172.17.0.2", "IPPrefixLen": 16, "IPv6Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "MacAddress": "02:42:ac:11:00:02", "DriverOpts": {} }, "mybridge": { "IPAMConfig": null, "Links": null, "Aliases": [ "ed3d8c95d32e" ], "NetworkID": "54415915c9f9277d4bf5114f0b1e055f72002cd0d49c80b4f16a505344721cd3", "EndpointID": "bdeee9ee6a0c4d80a963975e8e33e95e52de2b22580bc13581990cc36867e463", "Gateway": "172.20.0.1", "IPAddress": "172.20.0.2", "IPPrefixLen": 16, "IPv6Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "MacAddress": "02:42:ac:14:00:02", "DriverOpts": null } }让box1容器取消连接bridge网络docker network disconnect bridge box1自定义的bridge可以提供容器名称到容器IP地址的解析服务，例如可以直接ping box1bill@bill-lixiang-a:~$ docker container run -d --rm --name box2 --network=mybridge busybox /bin/sh -c "while true; do sleep 3600; done" bill@bill-lixiang-a:~$ docker exec -it box2 ping box1 PING box1 (172.20.0.2): 56 data bytes 64 bytes from 172.20.0.2: seq=0 ttl=64 time=0.061 ms 64 bytes from 172.20.0.2: seq=1 ttl=64 time=0.141 ms 64 bytes from 172.20.0.2: seq=2 ttl=64 time=0.138 ms 64 bytes from 172.20.0.2: seq=3 ttl=64 time=0.176 ms ^C --- box1 ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 0.061/0.129/0.176 ms创建网络时指定子网掩码和网关docker network create -d bridge --gateway=172.200.0.1 --subnet=172.200.0.0/16 demo查看demo bridge信息bill@bill-lixiang-a:~$ docker network inspect demo [ { "Name": "demo", "Id": "e5f71518d5114a504eacc16ad39f895f7e6849323fb531abf8334d5fd3668f25", "Created": "2022-08-24T19:48:12.210521782+08:00", "Scope": "local", "Driver": "bridge", "EnableIPv6": false, "IPAM": { "Driver": "default", "Options": {}, "Config": [ { "Subnet": "172.200.0.0/16", "Gateway": "172.200.0.1" } ] }, "Internal": false, "Attachable": false, "Ingress": false, "ConfigFrom": { "Network": "" }, "ConfigOnly": false, "Containers": {}, "Options": {}, "Labels": {} } ]容器的端口转发查看容器的IP地址docker container inspect --format '{{.NetworkSettings.IPAddress}}' box1Dockerfile内不写EXPOSE 端口号，也可以在容器创建/运行的时候，使用-p参数进行端口转发。EXPOSE更多的作用是给Docerfile的使用者以提示（a type of documentation）Docker host 网络详解# 删掉之前的实验box docker container rm -f box1 box2 # 新建box使用host网络 docker container run -d --rm --name box1 --network=host busybox /bin/sh -c "while true; do sleep 3600; done"host模式下建两个nginx containerdocker container run -d --name web1 --network=host nginx docker container run -d --name web2 --network=host nginxweb1启动成功，但web2启动失败，因为web1和web2均绑定了宿主机的80端口，web1已经使用了80端口，因此web2绑定失败Docker null网络docker container rm -f box1 # 新建box使用none网络 docker container run -d --rm --name box1 --network=none busybox /bin/sh -c "while true; do sleep 3600; done" docker container exec -it box1 sh / # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever / #只有lo本地网络，与外界无法通信Linux网络命名空间(未理解)Linux的Namespace（命名空间）技术是一种隔离技术，常用的Namespace有 user namespace, process namespace, network namespace等在Docker容器中，不同的容器通过Network namespace进行了隔离，也就是不同的容器有各自的IP地址，路由表等，互不影响。实验准备一台Linux机器，这一节会用到一个叫 brtcl 的命令，这个命令需要安装，如果是Ubuntu的系统，可以通过 apt-get install bridge-utils 安装；如果是Centos系统，可以通过 sudo yum install bridge-utils 来安装创建bridgebill@bill-lixiang-a:~$ sudo brctl addbr mydocker0 [sudo] bill 的密码： bill@bill-lixiang-a:~$ brctl show bridge name bridge id STP enabled interfaces br-165075680a9c 8000.02422a4ad8c0 no vetha63ea58 vethccd3776 br-1d0b2dcf23fc 8000.024209dfbea4 no veth38d5113 veth73c221d vethb57f9a8 vethe7eb813 vetheea0225 vetheef23fc br-54415915c9f9 8000.0242cdd97887 no br-e5f71518d511 8000.024276bbb280 no docker0 8000.024236471856 no veth76eb360 mydocker0 8000.000000000000 no准备一个shell脚本add-ns-to-br.sh#!/bin/bash bridge=$1 namespace=$2 addr=$3 vethA=veth-$namespace vethB=eth00-$namespace sudo ip netns add $namespace sudo ip link add $vethA type veth peer name $vethB sudo ip link set $vethB netns $namespace sudo ip netns exec $namespace ip addr add $addr dev $vethB sudo ip netns exec $namespace ip link set $vethB up sudo ip link set $vethA up sudo brctl addif $bridge $vethA脚本执行sh add-ns-to-br.sh mydocker0 ns1 172.16.1.1/16 sh add-ns-to-br.sh mydocker0 ns2 172.16.1.2/16把mydocker0这个bridge up起来sudo ip link set dev mydocker0 up验证[vagrant@docker-host1 ~]$ sudo ip netns exec ns1 bash [root@docker-host1 vagrant]# ip a 1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 5: eth00@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether f2:59:19:34:73:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 172.16.1.1/16 scope global eth00 valid_lft forever preferred_lft forever inet6 fe80::f059:19ff:fe34:7370/64 scope link valid_lft forever preferred_lft forever [root@docker-host1 vagrant]# ping 172.16.1.2 PING 172.16.1.2 (172.16.1.2) 56(84) bytes of data. 64 bytes from 172.16.1.2: icmp_seq=1 ttl=64 time=0.029 ms 64 bytes from 172.16.1.2: icmp_seq=2 ttl=64 time=0.080 ms ^C --- 172.16.1.2 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1000ms rtt min/avg/max/mdev = 0.029/0.054/0.080/0.026 ms [root@docker-host1 vagrant]#对外通信https://www.karlrupp.net/en/computer/nat_tutorial多容器应用部署实验Python Flask + Redis 练习使用自定义bridge，这样可以通过容器的名字去通信（DNS的效果），而默认bridge无此功能准备一个flask程序app.pyfrom flask import Flask from redis import Redis import os import socket app = Flask(__name__) redis = Redis(host=os.environ.get('REDIS_HOST', '127.0.0.1'), port=6379) @app.route('/') def hello(): redis.incr('hits') return f"Hello Container World! I have been seen {redis.get('hits').decode('utf-8')} times and my hostname is {socket.gethostname()}.\n"DockerfileFROM python:3.9.5-slim RUN pip install flask redis && \ groupadd -r flask && useradd -r -g flask flask && \ mkdir /src && \ chown -R flask:flask /src USER flask COPY app.py /src/app.py WORKDIR /src ENV FLASK_APP=app.py REDIS_HOST=redis EXPOSE 5000 CMD ["flask", "run", "-h", "0.0.0.0"]构建flask镜像，准备一个redis镜像docker pull redis docker build -t flask-demo .创建docker-bridgedocker network create -d bridge demo-network创建redis container、flask containerdocker container run -d --name redis-server --network demo-network redis docker container run -d --network demo-network --name flask-demo --env REDIS_HOST=redis-server -p 5000:5000 flask-demo打开浏览器访问 http://127.0.0.1:5000应该能看到类似下面的内容，每次刷新页面，计数加1Hello Container World! I have been seen 3 times and my hostname is d7c13e2fbe5f.总结：如果把上面的步骤合并到一起，成为一个部署脚本# prepare image docker image pull redis docker image build -t flask-demo . # create network docker network create -d bridge demo-network # create container docker container run -d --name redis-server --network demo-network redis docker container run -d --network demo-network --name flask-demo --env REDIS_HOST=redis-server -p 5000:5000 flask-demo第七章 Docker Compose什么是Docker Compose简化多容器部署安装Docker ComposeWindows和Mac在默认安装了docker desktop以后，docker-compose随之自动安装docker-compose --version最新版本号可以在这里查询 https://github.com/docker/compose/releases，可有从这里下载，添加环境变量也可以通过工具安装dnf install docker-compose # fedora yum install docker-compose # CentOS 7/ RHEL7 apt-get install docker-compose # debian及其变种如Ubuntu apk add docker-compose # alpine pacman -S docker-compose # arch安装完毕后查看版本号bill@bill-lixiang-a:~$ docker-compose --version docker-compose version 1.25.0, build unknowncompose文件的结构和版本docker compose文件的语法说明 https://docs.docker.com/compose/compose-file/基本语法结构version: "3.8" services: # 容器 servicename: # 服务名字，这个名字也是内部 bridge网络可以使用的 DNS name image: # 镜像的名字 command: # 可选，如果设置，则会覆盖默认镜像里的 CMD命令 environment: # 可选，相当于 docker run里的 --env volumes: # 可选，相当于docker run里的 -v networks: # 可选，相当于 docker run里的 --network ports: # 可选，相当于 docker run里的 -p servicename2: volumes: # 可选，相当于 docker volume create networks: # 可选，相当于 docker network create以上一章结尾的python+flask改造为例：docker image pull redis docker image build -t flask-demo . # create network docker network create -d bridge demo-network # create container docker container run -d --name redis-server --network demo-network redis docker container run -d --network demo-network --name flask-demo --env REDIS_HOST=redis-server -p 5000:5000 flask-demo改造成dokcer-compose.yml：version: "3.8" services: flask-demo: image: flask-demo:latest environment: - REDIS_HOST=redis-server networks: - demo-network ports: - 8080:5000 redis-server: image: redis:latest networks: - demo-network networks: demo-network:demo-network不指定driver的话默认为bridgedocker-compose 语法版本向后兼容https://docs.docker.com/compose/compose-file/在命令行使用docker-composedocker-compose up Creating network "flask_demo-network" with the default driver Creating flask_redis-server_1 ... done Creating flask_flask-demo_1 ... done Attaching to flask_flask-demo_1, flask_redis-server_1 redis-server_1 | 1:C 26 Aug 2022 06:23:51.210 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo redis-server_1 | 1:C 26 Aug 2022 06:23:51.210 # Redis version=7.0.4, bits=64, commit=00000000, modified=0, pid=1, just started redis-server_1 | 1:C 26 Aug 2022 06:23:51.210 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf redis-server_1 | 1:M 26 Aug 2022 06:23:51.210 * Increased maximum number of open files to 10032 (it was originally set to 1024). redis-server_1 | 1:M 26 Aug 2022 06:23:51.210 * monotonic clock: POSIX clock_gettime redis-server_1 | 1:M 26 Aug 2022 06:23:51.210 * Running mode=standalone, port=6379. redis-server_1 | 1:M 26 Aug 2022 06:23:51.210 # Server initialized redis-server_1 | 1:M 26 Aug 2022 06:23:51.210 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. redis-server_1 | 1:M 26 Aug 2022 06:23:51.211 * Ready to accept connections flask-demo_1 | * Serving Flask app 'app.py' flask-demo_1 | * Debug mode: off flask-demo_1 | WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. flask-demo_1 | * Running on all addresses (0.0.0.0) flask-demo_1 | * Running on http://127.0.0.1:5000 flask-demo_1 | * Running on http://172.22.0.3:5000 flask-demo_1 | Press CTRL+C to quit上面的过程是在前台运行在后台运行使用-d参数即可查看log类似docker查看某个容器的logdocker-compose logsdocker-compose logs -fdocker-compose up -ddocker-compose psdocker-compose stopdocker-compose rm通过docker container ls可以看到通过docker-compose创建的容器都有前缀可以通过指定容器前缀参数-p替换掉系统默认前缀docker-compose -p myproject up -d如果使用自定义前缀参数-p，在使用docker-compose进行其他操作时，都需要带上-pdocker-compose -p myproject psdocker-compose -p myproject stop docker-compose -p myproject rm容器名称指定version: "3.8" services: flask-demo: # 自定义容器名，但会影响到后面的scale操作 container_name: my-flask-demo image: flask-demo:latest environment: - REDIS_HOST=redis-server networks: - demo-network ports: - 8080:5000 redis-server: image: redis:latest networks: - demo-network networks: demo-network:docker-compose 镜像的拉取和构建docker-compose引用的镜像在本地不存在时，会去Docker Hub上自动拉取在docker-compose.yml中加入build参数可以在本地进行构建，提前准备好构建目录flask，与docker-compose.yml在同一目录使用docker-compose build进行构建. ├── docker-compose.yml └── flask ├── app.py └── Dockerfiledocker-compose.ymlversion: "3.8" services: flask-demo: build: ./flask image: flask-demo:latest environment: - REDIS_HOST=redis-server networks: - demo-network ports: - 8080:5000 redis-server: image: redis:latest networks: - demo-network networks: demo-network:docker-compose build# 先删除本地的镜像 docker image rm flask-demo:latest # build docker-compose build redis-server uses an image, skipping Building flask-demo Sending build context to Docker daemon 3.072kB Step 1/8 : FROM python:3.9.5-slim ---> c71955050276 Step 2/8 : RUN pip install flask redis && groupadd -r flask && useradd -r -g flask flask && mkdir /src && chown -R flask:flask /src ---> Running in 34a10a3c414b Collecting flask Downloading Flask-2.2.2-py3-none-any.whl (101 kB) Collecting redis Downloading redis-4.3.4-py3-none-any.whl (246 kB) Collecting Jinja2>=3.0 Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB) Collecting Werkzeug>=2.2.2 Downloading Werkzeug-2.2.2-py3-none-any.whl (232 kB) Collecting click>=8.0 Downloading click-8.1.3-py3-none-any.whl (96 kB) Collecting itsdangerous>=2.0 Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB) Collecting importlib-metadata>=3.6.0 Downloading importlib_metadata-4.12.0-py3-none-any.whl (21 kB) Collecting zipp>=0.5 Downloading zipp-3.8.1-py3-none-any.whl (5.6 kB) Collecting MarkupSafe>=2.0 Downloading MarkupSafe-2.1.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB) Collecting deprecated>=1.2.3 Downloading Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB) Collecting packaging>=20.4 Downloading packaging-21.3-py3-none-any.whl (40 kB) Collecting async-timeout>=4.0.2 Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB) Collecting wrapt<2,>=1.10 Downloading wrapt-1.14.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (77 kB) Collecting pyparsing!=3.0.5,>=2.0.2 Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB) Installing collected packages: zipp, wrapt, pyparsing, MarkupSafe, Werkzeug, packaging, Jinja2, itsdangerous, importlib-metadata, deprecated, click, async-timeout, redis, flask Successfully installed Jinja2-3.1.2 MarkupSafe-2.1.1 Werkzeug-2.2.2 async-timeout-4.0.2 click-8.1.3 deprecated-1.2.13 flask-2.2.2 importlib-metadata-4.12.0 itsdangerous-2.1.2 packaging-21.3 pyparsing-3.0.9 redis-4.3.4 wrapt-1.14.1 zipp-3.8.1 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv WARNING: You are using pip version 21.1.3; however, version 22.2.2 is available. You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command. Removing intermediate container 34a10a3c414b ---> a33a8f69c0d3 Step 3/8 : USER flask ---> Running in aac585018a53 Removing intermediate container aac585018a53 ---> bf6ae47bf7ec Step 4/8 : COPY app.py /src/app.py ---> 8a001223aa26 Step 5/8 : WORKDIR /src ---> Running in a48772836061 Removing intermediate container a48772836061 ---> 97f70b13d3c6 Step 6/8 : ENV FLASK_APP=app.py REDIS_HOST=redis ---> Running in a357a0a187de Removing intermediate container a357a0a187de ---> 4538f84aa7fa Step 7/8 : EXPOSE 5000 ---> Running in 43ccfcaa47d6 Removing intermediate container 43ccfcaa47d6 ---> 931950029385 Step 8/8 : CMD ["flask", "run", "-h", "0.0.0.0"] ---> Running in c23cb1f2a107 Removing intermediate container c23cb1f2a107 ---> 1a24120de6c1 Successfully built 1a24120de6c1 Successfully tagged flask-demo:latestbuild的更多参数重命名Dockerfilemv ./flask/Dockerfile ./flask/Dockerfile.dev设置build参数version: "3.8" services: flask-demo: build: context: ./flask dockerfile: Dockerfile.dev image: flask-demo:latest environment: - REDIS_HOST=redis-server networks: - demo-network ports: - 8080:5000 redis-server: image: redis:latest networks: - demo-network networks: demo-network:构建bill@bill-lixiang-a:~/flask$ docker-compose build redis-server uses an image, skipping Building flask-demo Sending build context to Docker daemon 3.072kB Step 1/8 : FROM python:3.9.5-slim ---> c71955050276 Step 2/8 : RUN pip install flask redis && groupadd -r flask && useradd -r -g flask flask && mkdir /src && chown -R flask:flask /src ---> Using cache ---> a33a8f69c0d3 Step 3/8 : USER flask ---> Using cache ---> bf6ae47bf7ec Step 4/8 : COPY app.py /src/app.py ---> Using cache ---> 8a001223aa26 Step 5/8 : WORKDIR /src ---> Using cache ---> 97f70b13d3c6 Step 6/8 : ENV FLASK_APP=app.py REDIS_HOST=redis ---> Using cache ---> 4538f84aa7fa Step 7/8 : EXPOSE 5000 ---> Using cache ---> 931950029385 Step 8/8 : CMD ["flask", "run", "-h", "0.0.0.0"] ---> Using cache ---> 1a24120de6c1 Successfully built 1a24120de6c1 Successfully tagged flask-demo:latestdocker-compose pulldocker-compose pull有的镜像需要在本地先去builddocker-compose 服务更新项目的文件有修改需要进行docker-compose服务更新使用--build参数，即可实现镜像更新docker-compose up -d --build总结使用--remove-orphans修改镜像文件，需要重新构建imagedocker-compose up -d --build使用restartdocker-compose 网络自己创建的driver为bridge的网络，支持容器名=>IP地址解析（DNS解析）docker-compose up后会自己建一个network(bridge)实验一、实验二、imap可以指定子网docker-compose 水平扩展和负载均衡实验环境及文件目录机构flask1 ├── docker-compose.yml └── flask ├── app.py └── Dockerfile 文件源码docker-compose.ymlversion: "3.8" services: flask: build: context: ./flask dockerfile: Dockerfile image: flask-demo:latest environment: - REDIS_HOST=redis-server redis-server: image: redis:latest client: image: xiaopeng163/net-box:latest command: sh -c "while true; do sleep 3600; done;"flask/app.pyfrom flask import Flask from redis import Redis import os import socket app = Flask(__name__) redis = Redis(host=os.environ.get('REDIS_HOST', '127.0.0.1'), port=6379) @app.route('/') def hello(): redis.incr('hits') return f"Hello Container World! I have been seen {redis.get('hits').decode('utf-8')} times and my hostname is {socket.gethostname()}.\n"flask/DockerfileFROM python:3.9.5-slim RUN pip install flask redis && \ groupadd -r flask && useradd -r -g flask flask && \ mkdir /src && \ chown -R flask:flask /src USER flask COPY app.py /src/app.py WORKDIR /src ENV FLASK=app.py REDIS_HOST=redis EXPOSE 5000 CMD ["flask", "run", "-h", "0.0.0.0"]启动及验证docker-compose up -d docker-compose ps进入到client中，测试docker-compose新建的bridge网络的容器名=>IP解析功能docker container ls | grep flask1docker container exec -it b504 sh /omd # ping flask /omd # ping redis-server可以ping通在client访问flask服务/omd # curl flask:5000 Hello Container World! I have been seen 1 times and my hostname is 6efbbf18a04a. /omd # curl flask:5000 Hello Container World! I have been seen 2 times and my hostname is 6efbbf18a04a.flask服务及redis服务测试访问成功，验证通过。flask服务水平扩展docker-compose up -d --scale flask=3 # 扩展总数# 向下scale docker-compose up -d --scale flask=1 # 扩展总数水平扩展后，会自动分配IP地址，如果去ping flask，docker网络可以自动负载均衡比如去扩展3个flask服务:docker-compose up -d --scale flask=3 # 进入到client容器 docker container exec -it b504 sh # 查看flask域名解析 /omd # nslookup flask Server: 127.0.0.11 Address: 127.0.0.11#53 Non-authoritative answer: Name: flask Address: 172.24.0.6 Name: flask Address: 172.24.0.5 Name: flask Address: 172.24.0.3可以看到flask对应三个address在client容器内访问flask服务也可以观察到负载均衡过程/omd # curl flask:5000 Hello Container World! I have been seen 3 times and my hostname is bfaf0a35033c. /omd # curl flask:5000 Hello Container World! I have been seen 4 times and my hostname is bfaf0a35033c. /omd # curl flask:5000 Hello Container World! I have been seen 5 times and my hostname is bfaf0a35033c. /omd # curl flask:5000 Hello Container World! I have been seen 6 times and my hostname is a28ccc40a7b4. /omd # curl flask:5000 Hello Container World! I have been seen 7 times and my hostname is a28ccc40a7b4. /omd # curl flask:5000 Hello Container World! I have been seen 8 times and my hostname is bfaf0a35033c. /omd # curl flask:5000 Hello Container World! I have been seen 9 times and my hostname is a28ccc40a7b4. /omd # curl flask:5000 Hello Container World! I have been seen 10 times and my hostname is a28ccc40a7b4. /omd # curl flask:5000 Hello Container World! I have been seen 11 times and my hostname is a28ccc40a7b4. /omd # curl flask:5000 Hello Container World! I have been seen 12 times and my hostname is a28ccc40a7b4. /omd # curl flask:5000 Hello Container World! I have been seen 13 times and my hostname is bfaf0a35033c. /omd # curl flask:5000 Hello Container World! I have been seen 14 times and my hostname is 6efbbf18a04a. /omd # curl flask:5000 Hello Container World! I have been seen 15 times and my hostname is a28ccc40a7b4. /omd # curl flask:5000 Hello Container World! I have been seen 16 times and my hostname is 6efbbf18a04a. /omd # curl flask:5000 Hello Container World! I have been seen 17 times and my hostname is bfaf0a35033c. /omd # curl flask:5000 Hello Container World! I have been seen 18 times and my hostname is a28ccc40a7b4. /omd # curl flask:5000 Hello Container World! I have been seen 19 times and my hostname is 6efbbf18a04a.可以看到请求被负载到不同的容器中整合Nginx负载均衡+Flask+Redis新的实验环境如下：flask2 ├── docker-compose.yml ├── flask │ ├── app.py │ └── Dockerfile └── nginx └── nginx.conf 文件源码ngxin反向代理配置proxy_pass http://flask:5000;使用自建的bridge的域名解析服务，结合水平扩展，实现自动负载均衡docker-compose.ymlversion: "3.8" services: flask: build: context: ./flask dockerfile: Dockerfile image: flask-demo:latest environment: - REDIS_HOST=redis-server networks: - backend - frontend redis-server: image: redis:latest networks: - backend nginx: image: nginx:stable-alpine ports: - 8000:80 depends_on: - flask volumes: - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf:ro - ./var/log/nginx:/var/log/nginx networks: - frontend networks: # 连接flask与redis backend: # 连接nginx与flask frontend:.dockerignore.DS_Store .idea .git .gitignore .env .dockerignore Dockerfile docker-compose.yaml nginx.conf varflask/app.pyfrom flask import Flask from redis import Redis import os import socket app = Flask(__name__) redis = Redis(host=os.environ.get('REDIS_HOST', '127.0.0.1'), port=6379) @app.route('/') def hello(): redis.incr('hits') return f"Hello Container World! I have been seen {redis.get('hits').decode('utf-8')} times and my hostname is {socket.gethostname()}.\n" flask/DockerfileFROM python:3.9.5-slim RUN pip install flask redis && \ groupadd -r flask && useradd -r -g flask flask && \ mkdir /src && \ chown -R flask:flask /src USER flask COPY app.py /src/app.py WORKDIR /src ENV FLASK=app.py REDIS_HOST=redis EXPOSE 5000 CMD ["flask", "run", "-h", "0.0.0.0"]nginx/nginx.confserver { listen 80 default_server; location / { proxy_pass http://flask:5000; } }启动服务docker-compose up -d访问：http://10.248.192.137:8000/水平扩展3个flask服务docker-compose up -d --scale flask=3 # 重启nginx服务 docker-compose restart nginx在访问http://10.248.192.137:8000/，多次刷新，查看负载均衡效果请求被负载均衡到3个不同的容器上docker-compose 环境变量实验环境基于上一节的整合Nginx负载均衡+Flask+Redis的实验源码修改docker-compose.yml，给redis增加连接密码修改app.py，增加redis连接密码docker-compose.ymlversion: "3.8" services: flask: build: context: ./flask dockerfile: Dockerfile image: flask-demo:latest environment: - REDIS_HOST=redis-server - REDIS_PASS=abc123 networks: - backend - frontend redis-server: image: redis:latest command: redis-server --requirepass abc123 networks: - backend nginx: image: nginx:stable-alpine ports: - 8000:80 depends_on: - flask volumes: - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf:ro - ./var/log/nginx:/var/log/nginx networks: - frontend networks: # 连接flask与redis backend: # 连接nginx与flask frontend: app.pyfrom flask import Flask from redis import Redis import os import socket app = Flask(__name__) redis = Redis(host=os.environ.get('REDIS_HOST', '127.0.0.1'), port=6379, password=os.environ.get('REDIS_PASS')) @app.route('/') def hello(): redis.incr('hits') return f"Hello Container World! I have been seen {redis.get('hits').decode('utf-8')} times and my hostname is {socket.gethostname()}.\n" 重新构建镜像并启动服务docker-compose build docker-compose up -d访问http://10.248.192.137:8000/连接redis成功redis密码被固定在了docker-compose.yml中，不安全，不通用一般来说redis密码需要作为服务的启动参数传入改造docker-compose.ymlversion: "3.8" services: flask: build: context: ./flask dockerfile: Dockerfile image: flask-demo:latest environment: - REDIS_HOST=redis-server - REDIS_PASS=${REDIS_PASSWORD} networks: - backend - frontend redis-server: image: redis:latest command: redis-server --requirepass ${REDIS_PASSWORD} networks: - backend nginx: image: nginx:stable-alpine ports: - 8000:80 depends_on: - flask volumes: - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf:ro - ./var/log/nginx:/var/log/nginx networks: - frontend networks: # 连接flask与redis backend: # 连接nginx与flask frontend: 将固定的redis密码abc123替换成${REDIS_PASSWORD}增加.env文件（与docker-compose.yml同级），并将.env加入到.dockerignore、.gitignore中，保护配置密码REDIS_PASSWORD=abc123校验配置docker-compose config可以看到redis密码变量被替换成.env文件中的配置也可以自己指定.env的文件名，比如新建myenv文件：REDIS_PASSWORD=ABC123在校验和启动时，带上--env-file参数docker-compose --env-file=myenv config可以看到ABC123注入成功自定义配置文件启动服务docker-compose --env-file=myenv up -d服务依赖和健康检查服务依赖通过depends_on配置项例如上面的整合Nginx负载均衡+Flask+Redis示例flask依赖于redis-servernginx依赖于flask首先启动redis、再启动flask、最后启动nginx光有服务依赖，还需要进行健康检查健康检查仍然使用上节课使用的flask程序先从Dockerfile的健康检查入手flask/DockerfileFROM python:3.9.5-slim RUN pip install flask redis && \ apt-get update && \ apt-get install -y curl && \ groupadd -r flask && useradd -r -g flask flask && \ mkdir /src && \ chown -R flask:flask /src USER flask COPY app.py /src/app.py WORKDIR /src ENV FLASK=app.py REDIS_HOST=redis EXPOSE 5000 HEALTHCHECK --interval=30s --timeout=30s \ CMD curl -f http://localhost:5000/ || exit 1 CMD ["flask", "run", "-h", "0.0.0.0"]加入HEALTHCHECK配置项容器安装curl进到flask目录，构建新镜像docker image build -t flask-demo . # 创建网络（这里没有用到docker-compose故需要自己创建bridge网络） docker network create mybridge docker container run -d --network mybridge --env REDIS_PASS=abc123 flask-demo docker container ls可以看到刚启动时，健康状态为health: starting，过了3分钟，显示如下：健康状态为：unhealthy，这是因为没有打开Redis的原因。docker container run -d --network mybridge --name redis redis:latest redis-server --requirepass abc123过30s，可以看到容器健康检测状态为healthy接下来通过修改docker-compose来实现服务的健康检查和依赖关系删除手动创建的flask、redis容器docker container rm -f 9c15 0842还原flask的Dockerfile，仅增加curl安装命令flask/DockerfileFROM python:3.9.5-slim RUN pip install flask redis && \ apt-get update && \ apt-get install -y curl && \ groupadd -r flask && useradd -r -g flask flask && \ mkdir /src && \ chown -R flask:flask /src USER flask COPY app.py /src/app.py WORKDIR /src ENV FLASK=app.py REDIS_HOST=redis EXPOSE 5000 CMD ["flask", "run", "-h", "0.0.0.0"]在docker-compose.yml中加入健康检测配置version: "3.8" services: flask: build: context: ./flask dockerfile: Dockerfile image: flask-demo:latest environment: - REDIS_HOST=redis-server - REDIS_PASS=${REDIS_PASSWORD} healthcheck: test: ["CMD", "curl", "-f", "http://localhost:5000"] interval: 30s timeout: 3s retries: 3 start_period: 40s depends_on: redis-server: condition: service_healthy networks: - backend - frontend redis-server: image: redis:latest command: redis-server --requirepass ${REDIS_PASSWORD} healthcheck: test: ["CMD", "redis-cli", "ping"] interval: 1s timeout: 3s retries: 30 networks: - backend nginx: image: nginx:stable-alpine ports: - 8000:80 depends_on: flask: condition: service_healthy volumes: - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf:ro - ./var/log/nginx:/var/log/nginx networks: - frontend networks: backend: frontend:运行docker-compose up -d可以看到启动等待顺序为redis->flask->nginx。docker-compose ps可以看到flask和redis的健康状态均为healthydocker compose 投票 app 练习源码地址： https://github.com/dockersamples/example-voting-appgit clone https://github.com/dockersamples/example-voting-app cd example-voting-app docker-compose up -d启动镜像后，查看运行的镜像列表：docker-compose ps投票端访问5000端口结果端访问5001端口第八章 Docker SwarmDocker Swarm介绍为什么不建议在生产环境中使用docker-Compose多机器如何管理？如果跨机器做scale横向扩展？容器失败退出时如何新建容器确保服务正常运行？如何确保零宕机时间？如何管理密码，Key等敏感数据？其它容器编排 swarmSwarm的基本架构swarm单节点快速上手激活单节点swarm环境# 初始化swarm集群，当前节点作为manager docker swarm init # 查看集群节点 docker node lssdf@cluster-System-Product-Name:~$ docker swarm init Swarm initialized: current node (zyhqfys9fw53hy3c8svszu9b0) is now a manager. To add a worker to this swarm, run the following command: docker swarm join --token SWMTKN-1-1klc3amj6w2fdd9hygqxgpkjfsi1aa6cqyqveq08q8ngsd7p0p-3cqe5wnu5kbtx213728dti4ar 192.168.3.76:2377 To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions. sdf@cluster-System-Product-Name:~$ docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION zyhqfys9fw53hy3c8svszu9b0 * cluster-System-Product-Name Ready Active Leader 20.10.16当服务器存在多个网卡时，例如我的VMware环境有双网卡，一个网卡用于固定IP，另一个网卡用于桥接主机网络连通外网。此时需要在docker swarm init时指定IP地址，例如在我的VMware虚拟机上:[bill@localhost ~]$ docker swarm init Error response from daemon: could not choose an IP address to advertise since this system has multiple addresses on different interfaces (192.168.72.131 on ens33 and 192.168.73.5 on ens36) - specify one with --advertise-addr此时需要使用--advertise-addr参数指定地址来初始化swarm[bill@localhost ~]$ docker swarm init --advertise-addr=192.168.73.5 Swarm initialized: current node (ddx18riw19jn17u01ruuv23w2) is now a manager. To add a worker to this swarm, run the following command: docker swarm join --token SWMTKN-1-33vl10uefiij94z6xmcupz42m83x6u3d8t6hsg38yfwd048of7-4dobqmik5z0ijbs3su21asdwa 192.168.73.5:2377 To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.docker swarm init 背后发生了什么主要是PKI和安全相关的自动化创建swarm集群的根证书manager节点的证书其它节点加入集群需要的tokens创建Raft数据库用于存储证书，配置，密码等数据RAFT相关资料http://thesecretlivesofdata.com/raft/https://raft.github.io/https://docs.docker.com/engine/swarm/raft/离开集群docker swarm leave [--force]sdf@cluster-System-Product-Name:~$ docker swarm Usage: docker swarm COMMAND Manage Swarm Commands: ca Display and rotate the root CA init Initialize a swarm join Join a swarm as a node and/or manager join-token Manage join tokens leave Leave the swarm unlock Unlock swarm unlock-key Manage the unlock key update Update the swarm Run 'docker swarm COMMAND --help' for more information on a command.swarm单节点serviceUsage: docker service create [OPTIONS] IMAGE [COMMAND] [ARG...]例如docker service create nginx:latest[bill@localhost ~]$ docker service create nginx:latest lh433gd5z8ten0qp9hif9uijr overall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged [bill@localhost ~]$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS lh433gd5z8te elegant_meitner replicated 1/1 nginx:latest [bill@localhost ~]$ docker service ps lh43 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PO RTS 5tjxnpl2p07q elegant_meitner.1 nginx:latest localhost.localdomain Running Running 4 minutes agolh433gd5z8ten0qp9hif9uijr是服务的ID，非容器的ID服务横向扩展[bill@localhost ~]$ docker service update lh43 --replicas 3 lh43 overall progress: 3 out of 3 tasks 1/3: running [==================================================>] 2/3: running [==================================================>] 3/3: running [==================================================>] verify: Service converged [bill@localhost ~]$ docker service ps lh43 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 5tjxnpl2p07q elegant_meitner.1 nginx:latest localhost.localdomain Running Running 7 minutes ago p5ekd0bhhj2t elegant_meitner.2 nginx:latest localhost.localdomain Running Running 44 seconds ago ze7jhtsoctkq elegant_meitner.3 nginx:latest localhost.localdomain Running Running 44 seconds ago [bill@localhost ~]$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS lh433gd5z8te elegant_meitner replicated 3/3 nginx:latestdocker service ps serviceID 是展示容器层面的信息，而docker service ls展示的是服务层面的信息删除服务docker service rm lh43swarm 三节点集群搭建下面的swarm集群实验均使用我的VMware虚拟机实验，虚拟机IP及配置如下固定IP配置名称192.168.73.52C（i5-11300H @ 3.10GHz）4Gnode1（manager）192.168.73.62C（i5-11300H @ 3.10GHz）4Gnode2（worker1）192.168.73.72C（i5-11300H @ 3.10GHz）4Gnode3（worker2）加入实验集群To add a worker to this swarm, run the following command: docker swarm join --token SWMTKN-1-33vl10uefiij94z6xmcupz42m83x6u3d8t6hsg38yfwd048of7-4dobqmik5z0ijbs3su21asdwa 192.168.73.5:2377多节点的环境涉及到机器之间的通信需求，所以防火墙和网络安全策略组是大家一定要考虑的问题，特别是在云上使用云主机的情况，下面这些端口记得打开 防火墙 以及 设置安全策略组TCP port 2376TCP port 2377TCP and UDP port 7946UDP port 4789TCP port 4567 pxc cluster 相互通讯端口为了简化，以上所有端口都允许节点之间自由访问就行。我的实验环境为centos7，故防火墙需要使用firewall，在三个节点上都要执行以下命令开放端口：sudo su - firewall-cmd --zone=public --add-port=2376/tcp --permanent firewall-cmd --zone=public --add-port=2377/tcp --permanent firewall-cmd --zone=public --add-port=7946/tcp --permanent firewall-cmd --zone=public --add-port=4789/tcp --permanent firewall-cmd --zone=public --add-port=4567/tcp --permanent firewall-cmd --reload exit在node1、node2上执行docker swarm joindocker swarm join --token SWMTKN-1-33vl10uefiij94z6xmcupz42m83x6u3d8t6hsg38yfwd048of7-4dobqmik5z0ijbs3su21asdwa 192.168.73.5:2377回到manager节点上，查看节点docker node ls由于两台虚拟机是由第一台虚拟机复制出来的，只修改了网卡的配置信息，hostname均相同，不利于分辨，故依次修改hostname，方便区分hostnamectl set-hostname manager hostnamectl set-hostname worker1 hostnamectl set-hostname worker2再次docker node ls，结构名称比较清晰swarm的overlay 网络创建使用overlay网络创建overlay网络docker network create -d overlay mynet到其他节点去查看网络docker network ls可以看到overlay网络也同步到了其他节点创建service并使用overlay网络docker service create --network mynet --name test --replicas 2 busybox ping 8.8.8.8查看service运行情况[bill@manager ~]$ docker service ps test ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS nx20if592mxm test.1 busybox:latest manager Running Running 26 seconds ago 6e30rufza9sg test.2 busybox:latest worker1 Running Running 25 seconds ago可以看到容器运行在两个不同的服务器上：manager、worker1进到worker1的busybox容器，查看IP和路由信息[bill@worker1 ~]$ docker container exec -it e61e sh / # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 16: eth0@if17: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue link/ether 02:42:0a:00:01:03 brd ff:ff:ff:ff:ff:ff inet 10.0.1.3/24 brd 10.0.1.255 scope global eth0 valid_lft forever preferred_lft forever 18: eth1@if19: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue link/ether 02:42:ac:13:00:03 brd ff:ff:ff:ff:ff:ff inet 172.19.0.3/16 brd 172.19.255.255 scope global eth1 valid_lft forever preferred_lft forever / # ip route default via 172.19.0.1 dev eth1 10.0.1.0/24 dev eth0 scope link src 10.0.1.3 172.19.0.0/16 dev eth1 scope link src 172.19.0.3swarm的overlay网络详解外部如何访问部署运行在swarm集群内的服务，即入方向流量，在swarm通过ingress来解决部署在swarm集群内的服务，如何对外进行访问，分为两块东西向流量，不同的swam节点上容器之间进行通信，swarm通过overlay网络解决南北向流量，swarm集群的容器如何对外访问，比如互联网，是通过Linux bridge和iptables NAT来解决的数据包从10.0.1.8发送到10.0.1.9时，会通过VXLAN协议进行封装，封装成从192.168.200.10发送到192.168.200.11的数据包swarm的ingress网络实验创建一个service，指定网络时overlay的mynet，通过-p映射端口使用containous/whoami是一个简单的web服务，能返回服务器的hostname，和基本的网络信息，比如IP地址docker service create --name web --network mynet -p 8080:80 --replicas 2 containous/whoami通过外部访问8080端口服务，会有一个负载均衡的效果[bill@manager ~]$ curl 192.168.73.5:8080 Hostname: e8090d83ebc7 IP: 127.0.0.1 IP: 10.0.0.20 IP: 172.19.0.5 IP: 10.0.1.30 RemoteAddr: 10.0.0.2:60914 GET / HTTP/1.1 Host: 192.168.73.5:8080 User-Agent: curl/7.29.0 Accept: */* [bill@manager ~]$ curl 192.168.73.5:8080 Hostname: 9296b889668a IP: 127.0.0.1 IP: 10.0.1.14 IP: 172.19.0.4 IP: 10.0.0.9 RemoteAddr: 10.0.0.2:60916 GET / HTTP/1.1 Host: 192.168.73.5:8080 User-Agent: curl/7.29.0 Accept: */*通过ipvs做了负载均衡关于这里的负载均衡这是一个stateless load balancing这是三层的负载均衡，不是四层的 LB is at OSI Layer 3 (TCP), not Layer 4 (DNS)以上两个限制可以通过Nginx或者HAProxy LB proxy解决 （https://docs.docker.com/engine/swarm/ingress/）swarm内部负载均衡和 VIP实验环境创建mynet的overlay网络，创建一个servicedocker service create --name web --network mynet --replicas 2 containous/whoami创建一个client的服务docker service create --name client --network mynet xiaopeng163/net-box:latest ping 8.8.8.8https://dockertips.readthedocs.io/en/latest/docker-swarm/internal_lb.html部署多service应用docker-compose用于本地开发测试stack用于生产环境手动部署略swarm stack部署多service应用先在swarm manager节点上安装一下docker-composesudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose sudo chmod +x /usr/local/bin/docker-compose模拟开发环境，编写代码，构建镜像git clone https://github.com/xiaopeng163/flask-redis cd flask-redis环境清理docker system prune -a -f编辑docker-compose.ymlimage修改为我的dockerhub id: billadayversion: "3.8" services: flask: build: context: ./ dockerfile: Dockerfile image: billaday/flask-redis:1.0 ports: - "8080:5000" environment: - REDIS_HOST=redis-server - REDIS_PASS=${REDIS_PASSWORD} redis-server: image: redis:latest command: redis-server --requirepass ${REDIS_PASSWORD}构建和查看镜像docker-compose build docker image ls提交镜像到dockerhub先登录dockerhub[bill@manager flask-redis]$ docker login Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. Username: billaday Password: Error response from daemon: Get "https://registry-1.docker.io/v2/": unauthorized: incorrect username or password [bill@manager flask-redis]$ docker login Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. Username: billaday Password: WARNING! Your password will be stored unencrypted in /home/bill/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded再推送镜像[bill@manager flask-redis]$ docker-compose push WARNING: The REDIS_PASSWORD variable is not set. Defaulting to a blank string. Pushing flask (billaday/flask-redis:1.0)... The push refers to repository [docker.io/billaday/flask-redis] ec90a017419f: Pushed ec184b69a655: Pushed 95557c4c07af: Mounted from library/python 873602908422: Mounted from library/python e4abe883350c: Mounted from library/python 95a02847aa85: Mounted from library/python b45078e74ec9: Mounted from library/python 1.0: digest: sha256:7d843ab99f01f0a6fec327e35371c0dc24111370db6a59d628b9db3e8b50af3d size: 1788可以看到镜像已经成功推送到dockerhub上通过stack启动服务（模拟生产环境）env REDIS_PASSWORD=ABC123 docker stack deploy --compose-file docker-compose.yml flask-demo使用--compose-file参数指定dockers-compose.yml文件[bill@manager flask-redis]$ docker stack ls NAME SERVICES ORCHESTRATOR flask-demo 2 Swarm参考资料Docker Tips (Docker笔记)