添加链接
link之家
链接快照平台
  • 输入网页链接,自动生成快照
  • 标签化管理网页链接

1. Rate-limiting 简介

Rate-limiting是kong 生态的一款开源限流插件, 提供根据时间窗口内请求数量限流的一款插件,其限流策略支持 Local Redis cluster , 限流粒度支持: consumer, credential, ip, service, header, path几个级别的限流, 限流方式可以用于 Global, Router, Service 。

Docker-compose.yml文件如下:

version: "3.7"
services: 
  kong-database:
    image: postgres:9.6
    container_name: kong-database
    ports:
      - "5432:5432"
    environment:
      - POSTGRES_USER=kong
      - POSTGRES_DB=kong
      - POSTGRES_PASSWORD=kongpass
    networks:
      - kong-net
    volumes:
      - ./docker-data/postgresql/postgresql:/var/lib/postgresql/data
  kong-gateway:
    # 镜像版本,目前最新
    image: kong/kong-gateway:2.8.1.0-alpine
    container_name: kong-gateway
    environment:
    # 数据持久化方式,使用postgres数据库
     - "KONG_DATABASE=postgres"
    # 数据库容器名称,Kong连接数据时使用些名称
     - "KONG_PG_HOST=kong-database"
     - "KONG_PG_USER=kong"
     - "KONG_PG_PASSWORD=kongpass"
    # 数据库名称
     - "KONG_CASSANDRA_CONTACT_POINTS=kong-database"
    # 日志记录目录
     - "KONG_PROXY_ACCESS_LOG=/dev/stdout"
     - "KONG_ADMIN_ACCESS_LOG=/dev/stdout"
     - "KONG_PROXY_ERROR_LOG=/dev/stderr"
     - "KONG_ADMIN_ERROR_LOG=/dev/stderr"
    # 暴露的端口
     - "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl"
     - "KONG_ADMIN_GUI_URL=http://localhost:8002"
    ports:
     - 8000:8000
     - 8443:8443
     - 8001:8001
     - 8444:8444
     - 8002:8002
     - 8445:8445
     - 8003:8003
     - 8004:8004
    # 使用docker网络
    networks:
     - kong-net
    # 依赖数据库服务
    depends_on:
     - kong-database
# kong 管理界面
  konga:
    image: pantsel/konga
    container_name: konga
    environment:
     - "TOKEN_SECRET=51liveup.cn"
     # 初始化使用development, 去初始化数据表, 正式使用production
     - "NODE_ENV=development"
     - "DB_ADAPTER=postgres" 
     - "DB_HOST=kong-database"
     - "DB_PORT=5432"
     - "DB_USER=kong"
     - "DB_PASSWORD=kongpass"
     - "DB_DATABASE=konga-db"
    ports:
     - 8080:1337
    networks:
     - kong-net
    depends_on:
     - kong-database
networks:
  kong-net:
    external: true

此文件执行启动时会包kong数据库初始化异常,需要执行初始化脚本,注意版本匹配

docker run --rm --network=kong-net  -e "KONG_DATABASE=postgres"  -e "KONG_PG_HOST=kong-database"  -e "KONG_PG_PASSWORD=kongpass" kong/kong-gateway:2.8.1.0-alpine kong migrations bootstrap

具体环境部署会在后期文章中详细讲解

下面进入正题,

因本次再使用Rate-limiting 插件中, 需要根据具体某一path 实现限流,其path需要支持通配, 故使用了konga UI 界面配置了 根据path限流, 如下

在测试过程中发现并未达到想要的效果, 即 path匹配则限流, path 不匹配则不限流, 在多次测试中发现, 均使用了Ip 限流,分析起源码发现是插件本身逻辑的问题

路径: /usr/local/share/lua/5.1/kong/plugins/rate-limiting

local function get_identifier(conf)
  local identifier
  if conf.limit_by == "service" then
    identifier = (kong.router.get_service() or
                  EMPTY).id
  elseif conf.limit_by == "consumer" then
    identifier = (kong.client.get_consumer() or
                  kong.client.get_credential() or
                  EMPTY).id
  elseif conf.limit_by == "credential" then
    identifier = (kong.client.get_credential() or
                  EMPTY).id
  elseif conf.limit_by == "header" then
    identifier = kong.request.get_header(conf.header_name)
  elseif conf.limit_by == "path" then
    local req_path = kong.request.get_path()
    if req_path == conf.path then
      identifier = req_path
  return identifier or kong.client.get_forwarded_ip()

逻辑解析: Rate limiting 只执行时需要根据 rout_id, service_id, indentifier, period_date, period生成一个唯一的local_key进行标识(如下图),而从上面的代码发现 indentifier是根你的限流粒度紧密相关,

从上面的代码可以看出在path匹配判断中, 如果path 匹配,将使用req_path作为identifier去后续生成local-key, 但是如果path不匹配,将使用ip 作为identifier, 故此时自动回退到使用ip 限流

local get_local_key = function(conf, identifier, period, period_date)
  local service_id, route_id = get_service_and_route_ids(conf)
  return fmt("ratelimit:%s:%s:%s:%s:%s", route_id, service_id, identifier,
             period_date, period)

2. 改造插件:

  • 由于我的部署方式采用的是docker 部署,故需要先将脚本文件挂载出来, 再修改,对docker-compose 配置文件 kong-gateway服务下添加挂载volumes,如下
  • volumes:
         - ./docker-data/kong/handler.lua:/usr/local/share/lua/5.1/kong/plugins/rate-limiting/handler.lua
    
  • 修改handler.lua脚本如下:
  • local function get_identifier(conf) local identifier if conf.limit_by == "service" then identifier = (kong.router.get_service() or EMPTY).id elseif conf.limit_by == "consumer" then identifier = (kong.client.get_consumer() or kong.client.get_credential() or EMPTY).id elseif conf.limit_by == "credential" then identifier = (kong.client.get_credential() or EMPTY).id elseif conf.limit_by == "header" then identifier = kong.request.get_header(conf.header_name) elseif conf.limit_by == "path" then local req_path = kong.request.get_path() --此处可以添加通配处理逻辑, if req_path == conf.path then identifier = req_path kong.client.get_forwarded_ip()) --去掉默认ip 限流配置,但此时如果path 不匹配 identifier 将会是空(nil) --return identifier or kong.client.get_forwarded_ip() return identifier
  • 修改此方法如下 local function get_usage(conf, identifier, current_timestamp, limits)
  • local function get_usage(conf, identifier, current_timestamp, limits)
      local usage = {}
      local stop
      --添加identifier 为空时不进行限流计算(源码没有这个为空校验),在后续处理中,stop为空,将不会进行限流
      if identifier then
    	  for period, limit in pairs(limits) do
    		local current_usage, err = policies[conf.policy].usage(conf, identifier, period, current_timestamp)
    		if err then
    		  return nil, nil, err
    		-- What is the current usage for the configured limit name?
    		local remaining = limit - current_usage
    		-- Recording usage
    		usage[period] = {
    		  limit = limit,
    		  remaining = remaining,
    		if remaining <= 0 then
    		  stop = period
      return usage, stop
    
  • 主方法处理逻辑如下:
  • function RateLimitingHandler:access(conf) local current_timestamp = time() * 1000 -- Consumer is identified by ip address or authenticated_credential id local identifier = get_identifier(conf) local fault_tolerant = conf.fault_tolerant -- Load current metric for configured period local limits = { second = conf.second, minute = conf.minute, hour = conf.hour, day = conf.day, month = conf.month, year = conf.year, local usage, stop, err = get_usage(conf, identifier, current_timestamp, limits) if err then if not fault_tolerant then return error(err) kong.log.err("failed to get usage: ", tostring(err)) --将限流信息放进header 中包含信息有: RATELIMIT_LIMIT, RATELIMIT_REMAINING,RATELIMIT_RESET等字段,可以在浏览器端查看 if usage then -- Adding headers local reset local headers if not conf.hide_client_headers then headers = {} local timestamps local limit local window local remaining for k, v in pairs(usage) do local current_limit = v.limit local current_window = EXPIRATION[k] local current_remaining = v.remaining if stop == nil or stop == k then current_remaining = current_remaining - 1 current_remaining = max(0, current_remaining) if not limit or (current_remaining < remaining) or (current_remaining == remaining and current_window > window) limit = current_limit window = current_window remaining = current_remaining if not timestamps then timestamps = timestamp.get_timestamps(current_timestamp) reset = max(1, window - floor((current_timestamp - timestamps[k]) / 1000)) headers[X_RATELIMIT_LIMIT[k]] = current_limit headers[X_RATELIMIT_REMAINING[k]] = current_remaining headers[RATELIMIT_LIMIT] = limit headers[RATELIMIT_REMAINING] = remaining headers[RATELIMIT_RESET] = reset -- If limit is exceeded, terminate the request -- 如果此处stop 为空将不会进行限流, if stop then headers = headers or {} headers[RETRY_AFTER] = reset return kong.response.error(429, "API rate limit exceeded", headers) if headers then kong.response.set_headers(headers) local ok, err = timer_at(0, increment, conf, limits, identifier, current_timestamp, 1) if not ok then kong.log.err("failed to create timer: ", err) 江湖十一郎 长方体混凝物质空间移动工程师 @大壮机砖厂
    粉丝