《Prometheus监控实战》第9章日志监控

link之家

链接快照平台

输入网页链接，自动生成快照
标签化管理网页链接

相关文章推荐

爱跑步的草稿纸 · “火锅界瑞幸”来了！海底捞前员工创业小火锅， ...· 1 年前 ·

愉快的铁链 · javascript - ...· 2 年前 ·

刚毅的皮带 · GridView动态创建列、多行表头_myg ...· 3 年前 ·

温文尔雅的皮蛋 · 北汽HOEN O2申报图曝光 ...· 3 年前 ·

开心的棒棒糖 · 废宅阿斗 NPM 即将被 Node.js ...· 3 年前 ·

腾讯云

备案控制台

云+社区

docker run -d --rm -p 9256:9256 --privileged -v /proc:/host/proc -v `pwd`:/config ncabatoff/process-exporter --procfs /host/proc -config.path /config/filename.yml

wget https://github.com/google/mtail/releases/download/v3.0.0-rc33/mtail_v3.0.0-rc33_linux_amd64 -0 mtail
chmod 0755 mtail
sudo cp mtail /usr/local/bin

mtail --version

sudo mkdir /etc/mtail

sudo touch /etc/mtail/line_count.mtail

counter line_count
/$/ {
  line_count++

/foo/ {
  ACTION1
} else {
  ACTION2

sudo mtail --progs /etc/mtail --logs '/var/log/*.log'

func Match
func Match(pattern, name string) (matched bool, err error)
Match reports whether name matches the shell file name pattern. The pattern syntax is:
pattern:
    { term }
term:
    '*'         matches any sequence of non-Separator characters
    '?'         matches any single non-Separator character
    '[' [ '^' ] { character-range } ']'
                character class (must be non-empty)
    c           matches character c (c != '*', '?', '\\', '[')
    '\\' c      matches character c
character-range:
    c           matches character c (c != '\\', '-', ']')
    '\\' c      matches character c
    lo '-' hi   matches character c for lo <= c <= hi
Match requires pattern to match all of name, not just a substring. The only possible returned error is ErrBadPattern, when pattern is malformed.
On Windows, escaping is disabled. Instead, '\\' is treated as path separator.

# Parser for the common apache "NCSA extended/combined" log format
# LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"
counter apache_http_requests_total by request_method, http_version, request_status
counter apache_http_bytes_total by request_method, http_version, request_status
/^/ +
/(?P<hostname>[0-9A-Za-z\.:-]+) / + # %h
/(?P<remote_logname>[0-9A-Za-z-]+) / + # %l
/(?P<remote_username>[0-9A-Za-z-]+) / + # %u
/\[(?P<timestamp>\d{2}\/\w{3}\/\d{4}:\d{2}:\d{2}:\d{2} (\+|-)\d{4})\] / + # %u
/"(?P<request_method>[A-Z]+) (?P<URI>\S+) (?P<http_version>HTTP\/[0-9\.]+)" / + # \"%r\"
/(?P<request_status>\d{3}) / + # %>s
/((?P<response_size>\d+)|-) / + # %b
/"(?P<referer>\S+)" / + # \"%{Referer}i\"
/"(?P<user_agent>[[:print:]]+)"/ + # \"%{User-agent}i\"
/$/ {
  strptime($timestamp, "02/Jan/2006:15:04:05 -0700") # for tests
  apache_http_requests_total[$request_method][$http_version][$request_status]++
  $response_size > 0 {
      apache_http_bytes_total[$request_method][$http_version][$request_status] += $response_size

counter apache_http_requests_total by request_method, http_version, request_status
counter apache_http_bytes_total by request_method, http_version, request_status

Constant pattern fragments
To re-use parts of regular expressions, you can assign them to a const identifier:
const PREFIX /^\w+\W+\d+ /
PREFIX {
  ACTION1
PREFIX + /foo/ {
  ACTION2
In this example, ACTION1 is done for every line that starts with the prefix regex, and ACTION2 is done for the subset of those lines that also contain 'foo'.
Pattern fragments like this don't need to be prefixes, they can be anywhere in the expression.
counter maybe_ipv4
const IPv4 /(?P<ip>\d+\.\d+\.\d+\.\d+)/
/something with an / + IPv4 + / address/ {
  maybe_ipv4++

(?P<request_status>\d{3})

Capture Groups
Regular expressions in patterns can contain capture groups -- subexpressions wrapped in parentheses. These can be referred to in the action block to extract data from the line being matched.
For example, part of a program that can extract from rsyncd logs may want to break down transfers by operation and module.
counter transfers_total by operation, module
/(?P<operation>\S+) (\S+) \[\S+\] (\S+) \(\S*\) \S+ (?P<bytes>\d+)/ {
  transfers_total[$operation][$3]++
Or, the value of the counter can be increased by the value of a capture group:
counter bytes_total by operation, module
/(?P<operation>\S+) (\S+) \[\S+\] (\S+) \(\S*\) \S+ (?P<bytes>\d+)/ {
  bytes_total[$operation][$3] += $bytes
Numeric capture groups address subexpressions in the match result as you might expect from regular expression groups in other languages, like awk and perl -- e.g. the expression $3 refers to the third capture group in the regular expression.
Named capture groups can be referred to by their name as indicated in the regular expression using the ?P<name> notation, as popularised by the Python regular expression library -- e.g. $bytes refers to (?P<bytes>\d+) in the examples above.
Capture groups can be used in the same expression that defines them, for example in this expression that matches and produces $x, then compares against that value.
/(?P<x>\d+)/ && $x > 1 {
  nonzero_positives++

{
  apache_http_requests_total[$request_method][$http_version][$request_status]++
  apache_http_bytes_total[$request_method][$http_version][$request_status] += $response_size

sudo mtail --progs /etc/mtail --logs '/var/log/apache/*.access'

counter rails_requests_started_total
counter rails_requests_started by verb
counter rails_requests_completed_total
counter rails_requests_completed by status
histogram rails_requests_completed_seconds by status buckets 0.005, 0.01, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 15.0
/^Started (?P<verb>[A-Z]+) .*/ {
  # Started HTTP requests by verb (GET, POST, etc.)
  rails_requests_started_total++
  rails_requests_started[$verb]++
/^Completed (?P<status>\d{3}) .+ in (?P<request_seconds>\d+)ms .*$/ {
  # Total numer of completed requests by status
  rails_requests_completed_total++
  rails_requests_completed[$status]++
  # Completed requests by status with histogram buckets
  # These statements "fall through", so the histogram is cumulative.  The
  # collecting system can compute the percentile bands by taking the ratio of
  # each bucket value over the final bucket.
  rails_requests_completed_seconds[$status] = $request_seconds / 1000.0

scrape_configs:
- job_name: 'mtail'
  file_sd_configs:
    - files:
      - targets/mtail/*.json
      refresh_interval: 5m

[{
  "targets": [
    "web:3903",
    "rails:3903"

腾讯云·社区

yeedomliu

《Prometheus监控实战》第9章 日志监控

《Prometheus监控实战》第9章 日志监控

第9章 日志监控

9.1 日志处理

9.2 mtail简介

9.2.1 安装mtail

9.2.2 使用mtail

9.2.3 运行mtail

9.3 处理Web服务器访问日志

9.4 解析Rails日志到直方图

9.5 部署mtail

9.6 抓取mtail端点

《Prometheus监控实战》第13章 监控Tornado

《Prometheus监控实战》第12章 监控Kubernetes

《Prometheus监控实战》第1章 监控简介

《Prometheus监控实战》第10章 探针监控

Prometheus:日志监控

《Prometheus监控实战》第8章 监控应用程序

《Prometheus监控实战》第2章 Prometheus简介

《Prometheus监控实战》第4章 监控主机和容器

Prometheus监控实战

《Prometheus监控实战》第13章监控Tornado

《Prometheus监控实战》第12章监控Kubernetes

《Prometheus监控实战》第1章　监控简介

《Prometheus监控实战》第10章探针监控

《Prometheus监控实战》第8章监控应用程序

《Prometheus监控实战》第2章　Prometheus简介

《Prometheus监控实战》第4章监控主机和容器