1、下载 Prometheus Server
Prometheus 下载与安装
下载地址:https://prometheus.io/download/
wget https://github.com/prometheus/prometheus/releases/download/v2.10.0/prometheus-2.10.0.linux-amd64.tar.gz
tar -xvf prometheus-2.10.0.linux-amd64.tar.gzmv prometheus-2.10.0.linux-amd64 /usr/local/prometheus210
cd /usr/local/prometheus210
可以最小配置启动 Prometheus服务,只用于测试。
./prometheus --config.file=prometheus.yml
2、创建systemd配置 与启动 Prometheus Server
创建 Prometheus 的 systemd 管理配置。
# vim /usr/lib/systemd/system/prometheus.service
[Unit]
Description=Prometheus
[Service]
Type=simple
ExecStart=/usr/local/prometheus210/prometheus --log.level=info \
--web.enable-lifecycle \
--web.enable-admin-api \
--storage.tsdb.path=/data1/prometheusTSDB/ \
--storage.tsdb.retention.time=90d \
--config.file=/usr/local/prometheus210/prometheus.yml
ExecReload=/bin/kill -s SIGHUP $MAINPID
[Install]
WantedBy=multi-user.target
配置说明:
- --log.level ,指定日志级别,Prometheus 会把日志写到标准输出上,在 /var/log/messages 能看到日志。
- --storage.tsdb.path ,指定 Prometheus TSDB 时序数据持久化的目录。
- --config.file ,指定 Prometheus Server 的配置文件。
- --web.enable-lifecycle ,想通过HTTP API接口 Reload Prometheus,需要在Prometheus启动时添加该参数启用这个功能。
- --web.enable-admin-api ,想通过HTTP API接口操作管理 Prometheus存储 TSDB,需要在Prometheus启动时添加该参数启用 TSDB Admin APIs 功能。
- --storage.tsdb.retention.time=90d ,数据保存周期,默认15天。
启动 Prometheus Server
mkdir -p /data1/prometheusTSDB
mkdir -p /data1/prometheusRULE
mkdir -p /data1/prometheusCONF
systemctl start prometheus
systemctl status prometheus
可以在浏览器打开访问:http://10.x.4.21:9090
3、配置 prometheus.yml 与 创建自定义metrics配置、报警发送规则
修改 Prometheus Server 主配置 prometheus.yml,配置 Alertmanager的地址,一个抓取 promethues自身metrics的 scrape配置。
[root@localhost prometheus210]# vim /usr/local/prometheus210/prometheus.yml
# my global config
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout is set to the global default (10s).
# Alertmanager configuration
alerting:
alertmanagers:
- static_configs:
- targets:
- 10.x.4.21:9093
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
- "/usr/local/prometheus210/metrics_custom.rules.yml"
- "/usr/local/prometheus210/send_alert.rules.yml"
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- job_name: 'prometheus'
# metrics_path defaults to '/metrics'
# scheme defaults to 'http'.
static_configs:
- targets: ['10.x.4.21:9090']
labels:
product: 'Email_person'
创建一个自定义 metrics的配置文件。
[root@localhost prometheus210]# vim /usr/local/prometheus210/metrics_custom.rules.yml
groups:
- name: example
rules:
- record: job_service:rpc_durations_seconds_count:avg_rate5m
expr: avg(rate(rpc_durations_seconds_count[5m])) by (job, service)
创建发送报警给 Alertmanager的规则文件。
[root@localhost prometheus210]# vim /usr/local/prometheus210/send_alert.rules.yml
groups:
- name: example
rules:
- alert: InstanceDown
expr: up == 0
for: 5m
labels:
severity: High
annotations:
summary: "Instance {{ $labels.instance }} down"
description: "{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 5 minutes."
- alert: RedisKey
expr: redis_db_keys > 5
for: 2m
labels:
severity: Info
annotations:
summary: "redis key greater than 5 on {{ $labels.instance }} "
description: "{{ $labels.instance }} current value is {{ $value }}"
重启 Prometheus Server
4、安装使用 Grafana 绘图【可选】
参考地址:https://grafana.com/grafana/download 或 http://docs.grafana.org/installation/rpm/
wget https://dl.grafana.com/oss/release/grafana-5.4.2-1.x86_64.rpm
sudo yum localinstall grafana-5.4.2-1.x86_64.rpm
安装后已经存在 grafana-server的 systemd管理配置。
ll /usr/lib/systemd/system/grafana-server.service
-rw-r--r-- 1 root root 1063 Oct 24 20:17 /usr/lib/systemd/system/grafana-server.service
systemctl start grafana-server
systemctl status grafana-server
初始密码是 admin/admin,登录改成 admin/123456 ,测试环境。
- 4.3、Grafana 配置Prometheus数据源
参考 http://docs.grafana.org/features/datasources/prometheus/