Openresty 集成 GeoIP2

时间：11-11来源：作者：点击数：25

一、openresty 安装配置

1、下载依赖

yum install -y gcc glibc gcc-c++ openssl-devel pcre-devel

2、下载解压 Openresty

官网下载地址：

https://openresty.org/cn/download.html

cd /data/softwares
wget -c https://openresty.org/download/openresty-1.15.8.3.tar.gz
tar xf openresty-1.15.8.3.tar.gz

3、安装 geoip2 动态识别库

使用geoip需要libmaxminddb对mmdb的高效访问，因此首先需要安装 libmaxminddb的动态识别库。

github下载地址:

https://github.com/maxmind/libmaxminddb/releases

安装步骤：

#1.下载解压
cd /data/softwares
wget -c https://github.com/maxmind/libmaxminddb/releases/download/1.4.3/libmaxminddb-1.4.3.tar.gz
tar xf libmaxminddb-1.4.3.tar.gz

#2.编译安装
cd libmaxminddb-1.4.3/
./configure 
make 
make install

默认情况下上述操作会将libmaxminddb.so部署到/usr/local/lib目录下，通过如下步骤更新ldconfig，可以让动态链接库为系统所共享。

echo /usr/local/lib >> /etc/ld.so.conf.d/local.conf
ldconfig

4、集成 nginx geoip2模块

github下载地址：

https://github.com/leev/ngx_http_geoip2_module/archive/

Openresty 集成：

cd /data/softwares
wget https://github.com/leev/ngx_http_geoip2_module/archive/3.3.tar.gz
tar xf 3.3.tar.gz
mv ngx_http_geoip2_module-3.3 ngx_http_geoip2_module

5、下载 GeoIP2 数据

官方下载地址

github下载地址

mkdir -p /data/softwares/GeoIP
git clone https://github.com/ar414-com/nginx-geoip2
cd nginx-geoip2
tar -zxvf GeoLite2-City_20200519.tar.gz
mv ./GeoLite2-City_20200519/GeoLite2-City.mmdb /data/softwares/GeoIP/
tar -zxvf GeoLite2-Country_20200519.tar.gz
mv ./GeoLite2-Country_20200519/GeoLite2-Country.mmdb /data/softwares/GeoIP/

6、编译启动openresty

编译安装：

cd /data/softwares/openresty-1.15.8.3/
./configure -j2 --prefix=/data/applications/openresty --with-pcre-jit --with-ipv6 --with-http_stub_status_module --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-http_sub_module --with-http_gzip_static_module --with-pcre --with-stream=dynamic --with-http_flv_module --add-module=/data/softwares/ngx_http_geoip2_module

make -j2
make install

配置环境变量

cat > /etc/profile.d/openresty.sh << EOF
export OPEN_HOME=/data/applications/openresty
export PATH=$OPEN_HOME/bin:$PATH
EOF

source /etc/profile.d/openresty.sh

启动openresty

openresty -V

openresty使用方法同nginx，只是命令由nginx替换为openresty

openresty

二、openresty 集成 geoip2

1、openresty 安装GeoIP2 Lua库

openresty包含了自身的包维护工具opm，该工具采用 perl实现依赖MD5，需要先安装perl的MD5模块。

yum install  -y perl-Digest-MD5

github地址：

https://github.com/anjia0532/lua-resty-maxminddb

opm安装lua api

opm get anjia0532/lua-resty-maxminddb

2、配置openresty nginx

需要在http段添加如下指令，其中的;;表示默认库路径：

lua_package_path "/usr/local/openresty/lualib/?.lua;;";

lua_package_cpath "/usr/local/openresty/lualib/?.so;;";

配置模版可参考：

user nginx;
worker_processes auto;
worker_cpu_affinity auto;
worker_rlimit_nofile 65535;

daemon on;
error_log /data/logs/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    use epoll;
    worker_connections  65535;
}

http {
    # 加载lua库和动态库
    lua_package_path  "/usr/local/openresty/lualib/?.lua;;";
    lua_package_cpath  "/usr/local/openresty/lualib/?.so;;";

    include mime.types;
    default_type application/octet-stream;

    charset utf-8;

    log_format main '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent" '
                    '"$http_x_forwarded_for" $host $request_time $upstream_response_time $request_body $scheme';

    log_format main1 '$remote_addr|$remote_user|[$time_local]|$request|'
                     '$status|$body_bytes_sent|$http_referer|'
                     '$http_user_agent|$request_time|$host|$upstream_addr|$request_body|$upstream_response_time';

    log_format lua '$remote_addr|$remote_user|[$time_local]|$request|'
                     '$status|$body_bytes_sent|$http_referer|'
                     '$http_user_agent|$request_time|$host|$upstream_addr|$request_id|$upstream_response_time';

    log_format main2
        '{"@timestamp":"$time_iso8601",'
        '"host":"$hostname",'
        '"server_ip":"$server_addr",'
        '"client_ip":"$http_x_forwarded_for",'
        '"xff":"$http_x_forwarded_for",'
        '"domain":"$host",'
        '"url":"$uri",'
        '"referer":"$http_referer",'
        '"args":"$args",'
        '"upstreamtime":"$upstream_response_time",'
        '"responsetime":"$request_time",'
        '"request_method":"$request_method",'
        '"status":"$status",'
        '"size":"$body_bytes_sent",'
        '"request_body":"$request_body",'
        '"request_length":"$request_length",'
        '"protocol":"$server_protocol",'
        '"upstreamhost":"$upstream_addr",'
        '"file_dir":"$request_filename",'
        '"http_user_agent":"$http_user_agent"'
     '}';

    # 基础优化设置
    server_tokens off;

    sendfile off;
    tcp_nopush on; 
    tcp_nodelay on;

    keepalive_timeout  65;
    keepalive_requests 8192;

    # gzip
    gzip on;
    gzip_min_length 1k;
    gzip_buffers 4 16k;
    gzip_comp_level 3;
    gzip_types text/plain application/javascript  text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/x-httpd-php image/jpeg image/gif image/png;
    gzip_vary on;

    client_body_timeout 600;
    client_header_timeout 600;
    send_timeout 600;
    reset_timedout_connection on;

    client_max_body_size 30m;
    client_body_buffer_size 8192k;

    client_header_buffer_size 16k;
    large_client_header_buffers 8 256k;

    server_names_hash_bucket_size 512;

    proxy_connect_timeout 600;
    proxy_read_timeout 600;
    proxy_send_timeout 600;

    proxy_buffer_size  128k;
    proxy_buffers   8 128k;
    proxy_busy_buffers_size 256k;

    output_buffers 1 32k;
    postpone_output 1460;

    open_file_cache max=65535 inactive=60s;
    open_file_cache_valid    80s;
    open_file_cache_min_uses 1;
    open_file_cache_errors   on;

    # fastcgi set
    fastcgi_ignore_client_abort       on;
    fastcgi_connect_timeout           600;
    fastcgi_send_timeout              600;
    fastcgi_read_timeout              600;
    fastcgi_buffer_size               128k;
    fastcgi_buffers                   8 128k;
    fastcgi_busy_buffers_size         256k;
    fastcgi_temp_file_write_size      256k;

    # fastcgi TEST
    fastcgi_cache_valid 200 302 1h;
    fastcgi_cache_valid 301 1d;
    fastcgi_cache_valid any 1m;
    fastcgi_cache_min_uses 1;
    fastcgi_cache_use_stale error timeout invalid_header http_500;

    real_ip_header X-Forwarded-For;

    # 开启缓存LUA代码(正式环境使用on)
    lua_code_cache off;

    # 允许用户自定义请求头
    underscores_in_headers on;

    # vhost
    include /data/conf/nginx/conf.d/*.conf;

    # nginx 使用 geoip配置（此为针对有负载均衡器时获取真实客户端IP）
    map $http_x_forwarded_for $real_ip {
        #~^(\d+\.\d+\.\d+\.\d+) $http_x_forwarded_for;
        #(?P)命名补货
        ~^(?P<firstAddr>[0-9\.]+),?.*$    $firstAddr;
        default $remote_addr;
    }

    # nginx加载使用geoip2数据库
    geoip2 /data/softwares/GeoIP/GeoLite2-City.mmdb {
        $geoip2_data_country_code source=$real_ip country iso_code;
        $geoip2_data_country_name source=$real_ip country names en;
        $geoip2_data_city_name source=$real_ip city names en;
        $geoip2_data_province_name subdivisions 0 names en;
        $geoip2_data_province_isocode subdivisions 0 iso_code;
    }

    # php变量设置
    fastcgi_param COUNTRY_CODE $geoip2_data_country_code;
    fastcgi_param COUNTRY_NAME $geoip2_data_country_name;
    fastcgi_param CITY_NAME    $geoip2_data_city_name;
    fastcgi_param PROVINCE_NMAE $geoip2_data_province_name;
}

3、编写lua脚本

1）nginx 虚拟主机配置

server {
    listen 80;
    server_name localhost;

    access_log /data/logs/nginx/status.access.log main2;
    error_log /data/logs/nginx/status.error.log error;

    # 获取geoip
    location / {
                default_type "text/html";
            charset utf-8;
                content_by_lua_file /data/conf/nginx/lua/geoip.lua;
    }

    location /myip {
        default_type "text/html";
        charset utf-8;
        content_by_lua_file /data/conf/nginx/lua/getip.lua;
    }
}

2）访问 mmdb lua 脚本内容如下

--- Generated by EmmyLua(https://github.com/EmmyLua)
------- Created by Tareya Shen.
------- DateTime: 2020/8/11 10:28 上午
------- Describe: lua 获取geoip

ngx.say("<br>IP location query result:<hr><br>")

local cjson=require 'cjson'
local geo=require 'resty.maxminddb'
local arg_ip=ngx.var.arg_ip
local arg_node=ngx.var.arg_node
ngx.say("IP:",arg_ip,", node:",arg_node,"<br>")
 
if not geo.initted() then
        geo.init("/data/softwares/GeoIP//GeoLite2-City.mmdb")
end
 
 
local res,err=geo.lookup(arg_ip or ngx.var.remote_addr)
 
if not res then
    ngx.say("Please check the ip address you provided: <div style='color:red'>",arg_ip,"</div>")
    ngx.log(ngx.ERR,' failed to lookup by ip , reason :',err)
else
    ngx.say("Result:",cjson.encode(res))
        if arg_node then
        ngx.say("node name:",ngx.var.arg_node, " , value:",cjson.encode(res[ngx.var.arg_node] or {}))
        end 
end

访问验证

IPv4

curl localhost/?ip=114.114.114.114&node=city

IPv6

curl localhost/?ip=2001:4860:0:1001::3004:ef68&node=country

3）获取客户端真是IP lua脚本

--- Generated by EmmyLua(https://github.com/EmmyLua)
------- Created by Tareya Shen.
------- DateTime: 2020/8/11 10:28 上午
------- Describe: 获取客户端真实IP

local clientIP = ngx.req.get_headers()["X-Real-IP"]

if clientIP == nil then
    clientIP = ngx.req.get_headers()["X-Forwarded-For"]
end

if clientIP == nil then
    clientIP = ngx.var.remote_addr
end

ngx.say(clientIP)