根据部署规划,如果对 Nginx 集群配置实现管理,需要在 GitLab、Jenkins 上完成相关的配置及编写 Ansible 剧本。本节将通过对配置文件 nginx.conf 举例 GitLab、Jenkins 及 Ansible 的配置,以实现 Nginx 配置管理的操作。
首先为 Nginx 配置创建用户及 Nginx 项目,操作步骤如下:
- git init
- git remote add origin http://IP:8080/nginx/homebox.git
- git add .
- git commit -m "Initial commit"
- git push -u origin master
根据 Nginx 配置目录的规划,定义 Ansible 剧本目录结构如下:
- .
- ├── ansible.cfg
- ├── hosts
- └── roles
- ├── nginx
- │ ├── defaults
- │ │ └── main.yaml
- │ ├── files
- │ │ ├── gzip.conf
- │ │ ├── fscgi.conf
- │ │ └── proxy.conf
- │ ├── handlers
- │ │ └── main.yaml
- │ ├── tasks
- │ │ ├── config_nginx.yaml
- │ │ ├── config_server.yaml
- │ │ ├── config_status.yaml
- │ │ ├── deploy.yaml
- │ │ ├── install.yaml
- │ │ ├── rollback.yaml
- │ │ └── main.yaml
- │ └── templates
- │ ├── nginx.conf
- │ ├── server.conf
- │ └── status.conf
- └── nginx.yaml
关于目录的说明如下所示:
- self_services: nginx
- exclude: ".git"
- rsync_opts:
- - "--exclude={{ exclude }}"
- process_events: >
- worker_processes auto;
- worker_rlimit_nofile 65535;
- worker_priority -5;
- modules: ""
- server: ""
- confdir: "/etc/nginx"
- env_packages:
- - pcre-devel
- - zlib-devel
- - openssl-devel
- - libxml2-devel
- - libxslt-devel
- - gd-devel
- - GeoIP-devel
- - jemalloc-devel
- - libatomic_ops-devel
- - luajit
- - luajit-devel
- - perl-devel
- - perl-ExtUtils-Embed
- # 重启Nginx服务任务
- - name: Restart Nginx services
- service:
- name: "{{ self_services }}"
- state: restarted
-
- # 启动Nginx服务任务
- - name: Start Nginx services
- service:
- name: "{{ self_services }}"
- state: started
- ---
- # 当变量deploy的值为deploy时执行deploy.yaml的任务步骤
- - name: "Starting deploy for nginx"
- include_tasks: deploy.yaml
- when: deploy == "deploy"
-
- # 当变量deploy的值为rollback时执行rollback.yaml的任务步骤
- - name: "Starting rollback for nginx"
- include_tasks: rollback.yaml
- when: deploy == "rollback"
- ---
- # 检查目标服务器是否存在配置文件,并将检查结果赋值给变量has_nginx
- - name: "check nginx service"
- stat: path={{ confdir }}/nginx.conf
- register: has_nginx
-
- # 如果目标服务器不存在Nginx服务器则调用分支任务install进行安装
- - name: "Starting install nginx "
- include_tasks: install.yaml
- when: not has_nginx.stat.exists
-
- # 如果当前任务为配置nginx.conf,则调用config_nginx任务配置nginx.conf文件
- - name: "Starting config nginx.conf "
- include_tasks: config_nginx.yaml
- when: not jobname == "" and jobname == "nginx.conf"
-
- # 如果当前任务为配置status.conf,则调用config_status任务配置status.conf文件
- - name: "Starting config website status for nginx"
- include_tasks: config_status.yaml
- when: not jobname == "" and jobname == "status.conf"
-
- # 如果当前任务为配置server.conf,则调用config_server任务配置server.conf文件
- - name: "Starting config website server for nginx"
- include_tasks: config_server.yaml
- when: not jobname == "" and jobname == "server.conf"
-
- # 初始化rsync模块的ssh免登录key
- - name: add authorized_keys
- authorized_key:
- user: "{{ ansible_user_id }}"
- key: "{{ lookup('file', '/home/jenkins/.ssh/id_rsa.pub') }}"
- state: present
- exclusive: no
-
- # 使用rsync模块将Nginx配置文件同步到目标机器
- - name: check rsync_opts rsync dir
- synchronize:
- src: "{{ work }}/"
- dest: "{{ confdir }}"
- delete: yes
- copy_links: yes
- private_key: "/home/jenkins/.ssh/id_rsa"
- rsync_opts: "{{ rsync_opts }}"
- register: rsync_result
-
- # 输出rsync的执行详情
- - debug: msg="{{ rsync_result.stdout_lines }}"
-
- # 使用Nginx的测试参数测试配置文件是否存在语法错误
- - name: Test Nginx Config
- shell: nginx -c {{ confdir }}/nginx.conf -t -q
- ignore_errors: True
- register: test_result
-
- # 如果执行检测失败,则停止当前任务,并输出检测结果
- - fail: msg="{{ test_result.stderr_lines }}"
- when: test_result.failed
-
- # 热加载Nginx进程
- - name: reload Nginx Service
- systemd: "name=nginx state=reloaded enabled=yes"
- ---
- - name: check rsync_opts rsync dir
- synchronize:
- src: "{{ work }}/"
- dest: "{{ confdir }}"
- delete: yes
- copy_links: yes
- private_key: "/home/jenkins/.ssh/id_rsa"
- rsync_opts: "{{ rsync_opts }}"
- register: rsync_result
-
- - debug: msg=" {{ rsync_result.stdout_lines }} "
-
- - name: "Test Nginx Config"
- shell: nginx -c {{ confdir }}/nginx.conf -t -q
- ignore_errors: True
- register: test_result
-
- - fail: msg="{{ test_result.stderr_lines }}"
- when: test_result.failed
-
- - name: reload Nginx Service
- systemd: "name=nginx state=reloaded enabled=yes"
- register: test_result
- ---
- # 通过模板文件与外部输入变量生成新的nginx.conf文件,替换Jenkins的工作目录中的
- # nginx.conf
- - name: "Starting init nginx.conf "
- template: src=nginx.conf dest={{ work }}/nginx.conf
- delegate_to: localhost
-
- # 因外部参数中的单、双引号及变量符号被转义,此处则重新替换回原符号
- - name: "Starting format nginx.conf "
- shell: sed -i 's/%24/$/g' {{ work }}/nginx.conf && sed -i 's/%9c/\"/g' {{ work }}/nginx.conf && sed -i "s/%98/\'/g" {{ work }}/nginx.conf && python /etc/ansible/bin/nginxfmt.py {{ work }}/nginx.conf
- delegate_to: localhost
- ---
- # 通过模板文件与外部输入变量生成新的虚拟主机文件,替换Jenkins的工作目录中虚拟主机
- # 文件并在conf.d目录下保存
- - name: "Starting init {{ jobname }} "
- template: src=server.conf dest={{ workdir }}/conf.d/{{ jobname }}.conf
- delegate_to: localhost
-
- # 因外部参数中的单、双引号及变量符号被转义,此处则重新替换回原符号
- - name: "Starting format nginx.conf "
- shell: sed -i 's/%24/$/g' {{ work }}/nginx.conf && sed -i 's/%9c/\"/g' {{ work }}/nginx.conf && sed -i "s/%98/\'/g" {{ work }}/nginx.conf && python /etc/ansible/bin/nginxfmt.py {{ work }}/nginx.conf
- delegate_to: localhost
- ---
- # 通过模板文件与外部输入变量生成新的状态监控虚拟主机文件,替换Jenkins的工作目录中
- # 的conf.d目录下保存
- - name: "Starting init status.conf "
- template: src=status.conf dest={{ workdir }}/conf.d/status.conf
- delegate_to: localhost
-
- # 因外部参数中的单、双引号及变量符号被转义,此处则重新替换回原符号
- - name: "Starting format nginx.conf "
- shell: sed -i 's/%24/$/g' {{ work }}/nginx.conf && sed -i 's/%9c/\"/g' {{ work }}/nginx.conf && sed -i "s/%98/\'/g" {{ work }}/nginx.conf && python /etc/ansible/bin/nginxfmt.py {{ work }}/nginx.conf
- delegate_to: localhost
- # 添加Nginx yum安装源
- - name: add repo
- yum_repository:
- name: nginx
- description: nginx repo
- baseurl: http://nginx.org/packages/centos/7/$basearch/
- gpgcheck: no
- enabled: 1
- # 安装环境依赖包
- - name: install centos packages
- yum:
- name: "{{ env_packages }}"
- disable_gpg_check: yes
- state: present
- # yum方式安装Nginx,并触发处理器Start Nginx services任务
- - name: install nginx
- yum:
- name: nginx
- state: latest
- notify: Start Nginx services
- {{ modules }}
- {{ process_events }}
- stream {
- {{ stream }}
- include conf.d/*.ream;
- }
- http {
- {{ http }}
-
- {% if gzip != "false" %}
- include gzip.conf; # HTTP gzip的配置文件
- {% endif %}
-
- {% if fscgi != "false" %}
- include fscgi.conf; # FastCGI代理的配置文件
- {% endif %}
-
- {% if proxy != "false" %}
- include proxy.conf; # HTTP代理配置
- {% endif %}
-
- include conf.d/*.conf;
- }
- {{ global }}
- upstream {
- {{ upstream }}
- }
- server{
- {{ server }}
- }
- {{ global }}
- server{
- {{ server }}
- }
- ---
- # 变量hosts由外部输入,设定操作的目标主机
- - hosts:
- - "{{ hosts }}"
- max_fail_percentage: 30 # 当有30%的操作目标任务执行出错时,则终止整个剧本的执行
- serial: "{{ serial }}" # 该模块可以设定操作目标数量实现灰度发布的效果,当设定为
- # 30%且操作目标为3台时,则表示一次仅操作一个目标
- roles:
- - nginx # 调用Nginx角色
-
根据 GitLab 及 Ansible 剧本的设置,Jenkins 需要创建具有如下操作内容的任务:
按照上述需求的设定,可以将不同的 Nginx 集群以文件夹类型任务进行创建,每个 Nginx 集群文件夹中包括 nginx.conf、status.conf 全局配置的自由风格任务,每个虚拟主机则按照虚拟主机名称创建自由风格任务分列在该集群文件夹下。任务层级结构如下:
- homebox # Nginx集群名称,任务类型为文件夹
- nginx.conf # nginx.conf任务,任务类型为自由风格
- status.conf # status.conf任务,任务类型为自由风格
- www.nginxbar.org # 虚拟主机任务,任务类型为自由风格
该任务层级设计,可以使操作者清晰地知道所操作的 Nginx 集群,同时还可以结合 Jenkins 的权限功能进行细粒度的权限控制。任务配置 nginx.conf 的创建步骤首先是在全局配置阶段通过参数化构建插件实现 Web 化变量的输入,通过参数化配置,设计部署与回滚操作选项。当选择回滚时,通过 Git 参数插件提供 Git 标签(tag)筛选功能列出可用的 Git 标签,选择后执行回滚操作。同时还要按照之前的规划在此阶段将 nginx.conf 文件内容分割成多个不同的变量,并定义为构建参数,让发布者在点击参数化构建后,可以通过 Web 界面进行选择和修改。
在构建操作配置阶段,编写 shell 脚本对所有输入的变量进行判断、修整后通过 ansible-playbook 命令传递给 Nginx 剧本,完成 Nginx 配置的修改、同步及加载操作。若在构建后动作配置阶段,则通过 Git Publisher 插件将当前的修改标记 Git 标签进行归档。详细配置过程如下。
- # 选择加载动态模块
- load_module "modules/ngx_http_geoip_module.so";
- load_module "modules/ngx_http_image_filter_module.so";
- load_module "modules/ngx_http_xslt_filter_module.so";
- # 工作进程及事件配置,定义文本参数process_events
- worker_processes auto; # 启动与CPU核数一致的工作进程
- worker_priority -5; # 工作进程在Linux系统中的优先级为-5
-
- events {
- worker_connections 65535; # 每个工作进程的最大连接数
- multi_accept on; # 每个工作进程每次都可以接受多个连接
- }
- # 配置TCP/UDP代理的日志格式模板,模板名为tcp
- log_format tcp '$remote_addr - $connection - [$time_local] $server_addr:$server_port - $protocol'
- '- $status - $upstream_addr - $bytes_received - $bytes_sent - $session_time '
- '- $proxy_protocol_addr:$proxy_protocol_port ';
-
- # 配置TCP/UDP代理的错误日志输出位置,错误级别为error
- error_log /var/log/nginx/tcp_error.log error;
- include mime.types; # 引入MIME类型映射表文件
-
- # 配置HTTP的错误日志输出位置,错误级别为error
- error_log /var/log/nginx/error.log error;
-
- # 配置HTTP的日志格式,模板名为main
- log_format main '$remote_addr - $connection - $remote_user [$time_local] "$request" - $upstream_addr '
- '$status - $body_bytes_sent - $request_time - "$http_referer" '
- '"$http_user_agent" - "$http_x_forwarded_for" - ';
-
- # 配置全局访问日志输出位置,并使用模板main的日志格式输出
- access_log /var/log/nginx/access.log main;
-
- charset utf-8; # 字符编码为utf-8
- variables_hash_max_size 2048; # 变量哈希表最大值为2048字节
- variables_hash_bucket_size 128; # 变量哈希桶最大值为128字节
- server_names_hash_bucket_size 256; # 服务主机名哈希桶大小为256字节
- client_header_buffer_size 32k; # 请求头缓冲区的大小为32KB
- large_client_header_buffers 4 128k; # 最大缓存为4个128KB
- client_max_body_size 20m; # 允许客户端请求的最大单个文件字节数为20MB
- sendfile on; # 开启零复制机制
- tcp_nopush on; # 启用在零复制时数据包最小传输的限制机制
- tcp_nodelay on; # 当处于保持连接状态时,以最快方式发送数据包
- keepalive_timeout 60; # 保持连接超时时间为60s
- client_header_timeout 10; # 读取客户请求头的超时时间是10s
- client_body_timeout 10; # 请求体接收超时时间为10s
- server_tokens on; # 不显示Nginx版本信息
编写构建脚本。
- #!/bin/bash
- set -x
-
- # 初始化变量
- jobname=${JOB_NAME}
- jobnum=${BUILD_TIMESTAMP}-${BUILD_NUMBER}
- OLD_IFS="$IFS" ;IFS="/" ;arr=($jobname) ;IFS="$OLD_IFS"
- cluster=${arr[1]}
- name=${arr[2]}
-
- # 部署时执行的操作
- if [ "$deploy" == "deploy" ];then
-
- rm -rf *.default
-
- # 对变量中的单引号、双引号及变量符号进行转义
- stream=${stream//$/%24}
- stream=${stream//\'/%98}
- stream=${stream//\"/%9c}
-
- http=${http//$/%24}
- http=${http//\'/%98}
- http=${http//\"/%9c}
-
- # 生成当前配置变量
- jobvars="process_events='$process_events' modules='$modules' stream= '$stream' http='$http' proxy='$proxy' gzip='$gzip' fscgi='$fscgi'"
-
- fi
-
- # 回滚时执行的操作
- if [ "$deploy" == "rollback" ];then
- OLD_IFS="$IFS" ;IFS="-" ;arr=($tag) ;IFS="$OLD_IFS"
- jobnum=${arr[${#arr[@]}-2]}-${arr[${#arr[@]}-1]}
- jobvars=""
- fi
-
- # 生成版本信息
- echo "#$cluster-$name-$jobnum $deploy by ${BUILD_USER}" >version.txt
-
- # 生成任务变量
- vars="hosts=$cluster jobname=$name work=${WORKSPACE} serial=30% deploy= '$deploy' $jobvars "
-
- # 执行Ansible剧本
- ansible-playbook -i /etc/ansible/hosts /etc/ansible/roles/nginx.yaml --extra-vars "$vars "
- if [ $? -ne 0 ];then exit 1; fi
-
- # 执行部署操作成功时,对变更的配置文件进行归档
- if [ "$deploy" == "deploy" ];then
- git add .
- git commit -m "#$cluster-$name-$jobnum deploy by ${BUILD_USER}"
- fi
添加修改构建名(Update build name),选择从文件名中读取(Read from file),文件名填写为 version.txt。
配置文件 staus.conf 及 server.conf 的 Jenkins 任务创建过程仅与 nginx.conf 在构建的参数配置和 shell 脚本上略有变化,此处就不一一详细举例了。
Jenkins 拥有诸多功能强大的插件,使其可以完成各种部署及发布的操作需求。例如,可以通过 jQuery 插件对 Jenkins 的操作界面进行自定义修改,增加根据选择项动态实现参数选项的显示和隐藏,或者增加自定义按钮实现配置预览等功能,此处就不再进行深入探讨了。结合 GitLab、Ansible 及 Jenkins 等开源软件,用户可以根据实际需求,不断优化并打造符合自身需求的 Nginx 配置管理工具。