centos7 安装logstash 7+
下载并安装公开签名密钥配置yum源安装logstash配置logstash获取nginx日志内容运行logstash
下载并安装公开签名密钥
GPG在Linux上的应用主要是实现官方发布的包的签名机制。 GPG分为公钥及私钥。 公钥：顾名思意，即可共享的密钥，主要用于验证私钥加密的数据及签名要发送给私钥方的数据。 私钥：由本地保留的密钥，用于签名本地数据及验证用公钥签名的数据。
实现原理(以Red Hat签名为例)： 1>RH在发布其官方的RPM包时（如本地RHEL光盘及FTP空间包），会提供一个GPG密钥文件，即所谓的公钥。 2>用户下载安装这个RPM包时，引入RH官方的这个RPM GPG公钥，用来验证RPM包是不是RH官方签名的。 导入GPG-KEY: 可以去https://www.redhat.com/security/team/key/或/etc/pki/rpm-gpg查找相应的GPG密钥，并导入到RPM: rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY
sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
如果出现error: https://artifacts.elastic.co/GPG-KEY-elasticsearch: import read failed(2)访问超时的方式 则用以下方法
sudo wget https://artifacts.elastic.co/GPG-KEY-elasticsearch --no-check-certificate
sudo rpm --import GPG-KEY-elasticsearch
配置yum源
vi /etc/yum.repos.d/logstash.repo
[logstash-7.x]
name=Elastic repository for 7.x packages
#baseurl=https://artifacts.elastic.co/packages/7.x/yum 因为国外站点速度很慢改用下面地址
baseurl=https://mirror.tuna.tsinghua.edu.cn/elasticstack/7.x/yum/
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
安装logstash
[root@master2 src]# sudo yum install logstash
Loaded plugins: fastestmirror, langpacks
base | 3.6 kB 00:00:00
elrepo | 2.9 kB 00:00:00
extras | 2.9 kB 00:00:00
logstash-7.x | 2.9 kB 00:00:00
updates | 2.9 kB 00:00:00
logstash-7.x/primary_db | 274 kB 00:00:02
Loading mirror speeds from cached hostfile
* base: ftp.sjtu.edu.cn
* elrepo: mirrors.tuna.tsinghua.edu.cn
* extras: mirrors.163.com
* updates: mirrors.163.com
Resolving Dependencies
--> Running transaction check
---> Package logstash.noarch 1:7.8.0-1 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=============================================================================================================================================================================================================================================
Package Arch Version Repository Size
=============================================================================================================================================================================================================================================
Installing:
logstash noarch 1:7.8.0-1 logstash-7.x 160 M
Transaction Summary
=============================================================================================================================================================================================================================================
Install 1 Package
Total download size: 160 M
Installed size: 160 M
Is this ok [y/d/N]:
Downloading packages:
logstash-7.8.0.rpm | 160 MB 00:00:12
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : 1:logstash-7.8.0-1.noarch 1/1
Using provided startup.options file: /etc/logstash/startup.options
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/pleaserun-0.0.31/lib/pleaserun/platform/base.rb:112: warning: constant ::Fixnum is deprecated
Successfully created system startup script for Logstash
Verifying : 1:logstash-7.8.0-1.noarch 1/1
Installed:
logstash.noarch 1:7.8.0-1
Complete!
软连接各安装目录到指定目录，日常规范，不是必要步骤
ln -s /etc/logstash/ /apps/conf/logstash
ln -s /usr/share/logstash/ /apps/svr/logstash
配置logstash获取nginx日志内容
cd /apps/conf/logstash/conf.d
vi nginx.conf
input{
file{
path => "/apps/logs/nginx/kibana-access.log"
codec => json
start_position => "beginning"
type => "nginx-log"
}
}
output{
stdout{
codec => rubydebug
}
}
运行logstash
直接用命令运行，进行配置文件检测
[root@master2 conf.d]# /apps/svr/logstash/bin/logstash -f /apps/conf/logstash/conf.d/nginx.conf --config.reload.automatic
{
"request_method" => "POST",
"request_time" => "0.055",
"request" => "POST /api/console/proxy?path=_mapping&method=GET HTTP/1.1",
"remote_user" => "-",
"http_referrer" => "http://kibana.liumaster.com/app/kibana",
"status" => "200",
"@version" => "1",
"http_x_forwarded_for" => "-",
"upstream_status" => "200",
"path" => "/apps/logs/nginx/kibana-access.log",
"type" => "nginx-log",
"host" => "kibana.liumaster.com",
"body_bytes_sent" => "21149",
"http_user_agent" => "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.113 Safari/537.36",
"remote_addr" => "192.168.254.1",
"@timestamp" => 2020-07-03T08:41:26.000Z,
"upstream_response_time" => "0.054"
}
正常输出，没问题。
新增输出至elasticsearch配置
input{
file{
path => "/apps/logs/nginx/kibana-access.log"
codec => json
start_position => "beginning"
type => "nginx-log"
}
}
output{
elasticsearch{
hosts => ["192.168.254.130:9200"]
user => 'elastic'
password => 'sHq5wTnRc08yrCcqU9gD'
index => "nginx-log-%{+YYYY.MM.dd}"
}
# stdout{
# codec => rubydebug
# }
}
设置开机启动
systemctl enable logstash.service
启动服务
systemctl start logstash.service
