Nginx 可以用 kill -HUP 来重启,不会丢失已建连接。Haproxy 如何做才能做到 zero downtime 无缝重载呢?
做法如下:
一、配置
编译harpoxy的时候带上参数 USE_SYSTEMD,选择 Haproxy 1.8 以上版本
1make TARGET=linux2628 USE_GETADDRINFO=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1 \
2 USE_SYSTEMD=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_NS=1
3make install
Haproxy无缝重载技术:
-
旧进程当前管理的连接根据 file descriptor 文件描述符通过 socket 套接字传输到新进程。
-
在这个过程中,文件socket(unix socket)的连接没有断开。
-
新进程在充当 master-worker 主工作者的同时执行此任务。
综上所述,通过使用unix socket来维护连接状态并在旧进程和新进程之间传递,防止了连接丢失。
haproxy 的运行使用 Systemd 重载,使用 -Ws 方式。 (此外,在构建时必须启用 USE_SYSTEMD)
1 -D : start as a daemon. The process detaches from the current terminal after
2 forking, and errors are not reported anymore in the terminal. It is
3 equivalent to the "daemon" keyword in the "global" section of the
4 configuration. It is recommended to always force it in any init script so
5 that a faulty configuration doesn't prevent the system from booting.
6
7 -W : master-worker mode. It is equivalent to the "master-worker" keyword in
8 the "global" section of the configuration. This mode will launch a "master"
9 which will monitor the "workers". Using this mode, you can reload HAProxy
10 directly by sending a SIGUSR2 signal to the master. The master-worker mode
11 is compatible either with the foreground or daemon mode. It is
12 recommended to use this mode with multiprocess and systemd.
13
14 -Ws : master-worker mode with support of `notify` type of systemd service.
15 This option is only available when HAProxy was built with `USE_SYSTEMD`
16 build option enabled.
具体的启动脚本:/etc/systemd/system/haproxy.service
1[Unit]
2Description=HAProxy Load Balancer
3After=network-online.target
4Wants=network-online.target
5
6[Service]
7Environment="CONFIG=/etc/haproxy/haproxy.cfg" "PIDFILE=/run/haproxy.pid"
8ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q
9ExecStart=/usr/sbin/haproxy -Ws -f $CONFIG -p $PIDFILE
10ExecReload=/usr/sbin/haproxy -f $CONFIG -c -q
11ExecReload=/bin/kill -USR2 $MAINPID
12KillMode=mixed
13Restart=always
14SuccessExitStatus=143
15Type=notify
16
17# The following lines leverage SystemD's sandboxing options to provide
18# defense in depth protection at the expense of restricting some flexibility
19# in your setup (e.g. placement of your configuration files) or possibly
20# reduced performance. See systemd.service(5) and systemd.exec(5) for further
21# information.
22
23# NoNewPrivileges=true
24# ProtectHome=true
25# If you want to use 'ProtectSystem=strict' you should whitelist the PIDFILE,
26# any state files and any other files written using 'ReadWritePaths' or
27# 'RuntimeDirectory'.
28# ProtectSystem=true
29# ProtectKernelTunables=true
30# ProtectKernelModules=true
31# ProtectControlGroups=true
32# If your SystemD version supports them, you can add: @reboot, @swap, @sync
33# SystemCallFilter=~@cpu-emulation @keyring @module @obsolete @raw-io
34
35[Install]
36WantedBy=multi-user.target
二、验证
验证是否真的是无缝重载的步骤如下:
在haproxy.cfg的global段落中加入stat的配置:
1stats socket /var/run/haproxy.sock level admin expose-fd listeners process 1
运行一个不断循环重启的脚本:
1while true ; do systemctl reload haproxy ; sleep 3 ; done
用 apache 的压测工具 ab 来压一下。
Send request while service is reloading:
1ab -r -c 20 -n 100000 http://127.0.0.1/
最后检查结果中 “Failed requests” 是否为零就可以了