Nginx 설정
1. nginx log 설정
- nginx 엑세스 로그 포맷에서는 “request_time” 옵션이 빠져있다.
- “request_time” metrics 수집을 위해서 설정해준다.
- log_format를 아래와 같이 수정해준다.
sudo vi /etc/nginx/nginx.conf
... 중간 생략 ...
http {
log_format main '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" "$request_time"';
access_log /var/log/nginx/access.log main;
... 중간 생략 ...
service nginx restart
2. nginx log 권한 설정
- nginx log exporter가 로그를 수집하기 위해서 로그 접근 권한 설정
sudo chmod 777 /var/log/nginx/access.log
2. Nginx log exporter 설정
1. nginx log exporter 설치
- nginx log exporter GitHub URL
- nginx log exporter 다운로드
cd ~
mkdir -p nginx_log
cd nginx_log
wget https://github.com/martin-helmich/prometheus-nginxlog-exporter/releases/download/v1.10.0/prometheus-nginxlog-exporter_1.10.0_linux_amd64.tar.gz
tar xvfz prometheus-nginxlog-exporter_1.10.0_linux_amd64.tar.gz
- nginx log exporter yaml 설정
sudo mkdir -p /etc/prometheus-nginxlog-exporter
sudo vi /etc/prometheus-nginxlog-exporter/prometheus-nginxlog-exporter.yml
listen:
port: 4040
metrics_endpoint: "/metrics"
consul:
enable: false
namespaces:
- name: nginx
format: "$remote_addr - $remote_user [$time_local] \"$request\" $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\" \"$request_time\""
source:
files:
- /var/log/nginx/access.log
only_count: true
relabel_configs:
- target_label: request_uri
from: request
split: 2
separator: ' '
sudo chown -R prometheus_nginxlog_exporter.prometheus_nginxlog_exporter /etc/prometheus-nginxlog-exporter/
- nginx log exporter 서비스 등록
sudo useradd -M -r -s /bin/false prometheus_nginxlog_exporter
sudo cp ./prometheus-nginxlog-exporter /usr/local/bin
sudo chown prometheus_nginxlog_exporter:prometheus_nginxlog_exporter /usr/local/bin/prometheus-nginxlog-exporter
sudo tee /etc/systemd/system/prometheus_nginxlog_exporter.service << EOF
[Unit]
Description=Prometheus Nginxlog Exporter
Wants=network-online.target
After=network-online.target
[Service]
User=prometheus_nginxlog_exporter
Group=prometheus_nginxlog_exporter
Type=simple
ExecStart=/usr/local/bin/prometheus-nginxlog-exporter -config-file /etc/prometheus-nginxlog-exporter/prometheus-nginxlog-exporter.yml
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl restart prometheus_nginxlog_exporter
sudo systemctl status prometheus_nginxlog_exporter
curl localhost:4040/metrics
# HELP nginx_http_response_count_total Amount of processed HTTP requests
# TYPE nginx_http_response_count_total counter
nginx_http_response_count_total{method="GET",request_uri="/metrics",status="200"} 4
# HELP nginx_http_response_size_bytes Total amount of transferred bytes
# TYPE nginx_http_response_size_bytes counter
nginx_http_response_size_bytes{method="GET",request_uri="/metrics",status="200"} 404
# HELP nginx_http_response_time_seconds Time needed by NGINX to handle requests
# TYPE nginx_http_response_time_seconds summary
nginx_http_response_time_seconds{method="GET",request_uri="/metrics",status="200",quantile="0.5"} 0
nginx_http_response_time_seconds{method="GET",request_uri="/metrics",status="200",quantile="0.9"} 0
nginx_http_response_time_seconds{method="GET",request_uri="/metrics",status="200",quantile="0.99"} 0
nginx_http_response_time_seconds_sum{method="GET",request_uri="/metrics",status="200"} 0
nginx_http_response_time_seconds_count{method="GET",request_uri="/metrics",status="200"} 4
# HELP nginx_http_response_time_seconds_hist Time needed by NGINX to handle requests
# TYPE nginx_http_response_time_seconds_hist histogram
nginx_http_response_time_seconds_hist_bucket{method="GET",request_uri="/metrics",status="200",le="0.005"} 4
nginx_http_response_time_seconds_hist_bucket{method="GET",request_uri="/metrics",status="200",le="0.01"} 4
nginx_http_response_time_seconds_hist_bucket{method="GET",request_uri="/metrics",status="200",le="0.025"} 4
nginx_http_response_time_seconds_hist_bucket{method="GET",request_uri="/metrics",status="200",le="0.05"} 4
nginx_http_response_time_seconds_hist_bucket{method="GET",request_uri="/metrics",status="200",le="0.1"} 4
nginx_http_response_time_seconds_hist_bucket{method="GET",request_uri="/metrics",status="200",le="0.25"} 4
nginx_http_response_time_seconds_hist_bucket{method="GET",request_uri="/metrics",status="200",le="0.5"} 4
nginx_http_response_time_seconds_hist_bucket{method="GET",request_uri="/metrics",status="200",le="1"} 4
nginx_http_response_time_seconds_hist_bucket{method="GET",request_uri="/metrics",status="200",le="2.5"} 4
nginx_http_response_time_seconds_hist_bucket{method="GET",request_uri="/metrics",status="200",le="5"} 4
nginx_http_response_time_seconds_hist_bucket{method="GET",request_uri="/metrics",status="200",le="10"} 4
nginx_http_response_time_seconds_hist_bucket{method="GET",request_uri="/metrics",status="200",le="+Inf"} 4
nginx_http_response_time_seconds_hist_sum{method="GET",request_uri="/metrics",status="200"} 0
nginx_http_response_time_seconds_hist_count{method="GET",request_uri="/metrics",status="200"} 4
# HELP nginx_parse_errors_total Total number of log file lines that could not be parsed
# TYPE nginx_parse_errors_total counter
nginx_parse_errors_total 0
# HELP prometheus_nginxlog_exporter_build_info A metric with a constant '1' value labeled by version, revision, branch, and goversion from which prometheus_nginxlog_exporter was built.
# TYPE prometheus_nginxlog_exporter_build_info gauge
prometheus_nginxlog_exporter_build_info{branch="v1.10.0",goversion="go1.18.6",revision="9274be256a47378a9930164023d16e1958e96292",version="1.10.0"} 1
3. prometheus scrape 설정
1. scrape 설정
- 폴더 위치 : prometheus/
- 파일 : values-mbx-dev.yaml
- 10.223.64.23:4040 추가
prometheus:
prometheusSpec:
additionalScrapeConfigs:
- job_name: "node-exporter-ec2"
scrape_interval: 10s
metrics_path: /metrics
static_configs:
- targets: ["10.223.64.23:9100","10.223.64.23:8080","10.223.64.23:4040"]
2. prometheus 재배포
helm upgrade --install prometheus . -f values-mbx-dev.yaml -n monitor