Docker搭建ELK 8.0，收集Nginx日志完整搭建教程——免踩坑系列

因公司节约成本，营业下云，良多根底办事都需要自建，刚好尝鲜一下ELK 8.0版本。详细搭建过程记录如下：

1、施行如下号令后，施行一下 sysctl -p 使设置装备摆设生效或者 reboot 重启一下办事器

echo "vm.max_map_count=262144" >> /etc/sysctl.conf 

2、在宿主机上新建elasticsearch数据存储目次并受权

mkdir -p /elasticsearch/data && chown 1000:1000 /elasticsearch/data

3、施行 docker-compose up -d 摆设elasticsearch和kibana，docker-compose.yaml如下：

version: '2.2'services: elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:8.6.1 container_name: elasticsearch restart: always environment: - discovery.tyPE=single-node - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms8g -Xmx8g" - TZ=PRC ulimits: memlock: soft: -1 hard: -1 volumes: - /elasticsearch/data:/usr/share/elasticsearch/data ports: - 9200:9200 kibana: image: docker.elastic.co/kibana/kibana:8.6.1 container_name: kibana restart: always environment: - server.publicBaseUrl=http://YOUr_server_ip:5601 depends_on: - elasticsearch ports: - 5601:5601

4、阅读器输入http://your_server_ip:5601看到如下界面申明启动一般

5、进入elasticsearch容器内重置elastic密码和生成Enrollment token，号令如下：

docker exec -it elasticsearch bash # 输入“y”后生成新的密码，需要保留记住bin/elasticsearch-reset-password -u elastic# 生成的Token密钥，填入到上图中的输入框内bin/elasticsearch-create-enrollment-token --scope kibana

6、点击阅读器的"Configure Elastic"按钮继续到下一步，如图所示：

7、进入kibana容器内获取6位数字验证码并填入框内，号令如下：

docker exec -it kibana bashbin/kibana-verification-code

8、设置装备摆设胜利后呈现下图，账号是：elastic ，密码是第5步中生成的密码

9、备份保留证书，后面需要用到。退出到容器外施行如下号令：

 docker cp elasticsearch:/usr/share/elasticsearch/config/certs/http_ca.crt .

10、将Nginx日记文件格局改成如下：

log_format log_json escape=json '{ "time": "$time_local", ' '"username": "$remote_user", ' '"remote_addr": "$remote_addr", ' '"request": "$request", ' '"domain": "$host", ' '"status": $status, ' '"size": $bytes_sent, ' '"request_body": "$request_body", ' '"client": "$http_user_agent", ' '"x_forwarded": "$http_x_forwarded_for", ' '"up_addr": "$upstream_addr", ' '"request_time": $request_time, ' '"upstream_response_time": $upstream_response_time, ' '"upstream_connect_time": $upstream_connect_time, ' '"upstream_header_time": $upstream_header_time, ' '"upstream_status": $upstream_status' ' }'; access_log /var/log/nginx/access.log log_json;

11、将http_ca.crt证书文件拷贝到Nginx办事器上，如/root/logstash/config/目次下，并修改用户权限：

chown 1000:1000 http_ca.crt

12、在/root/logstash/config/目次下新建logstash.conf设置装备摆设文件，内容如下：

input { file { path => ["/nginx/logs/*.access.log"] codec => "json" type => "nginx-access-log" } }filter { mutate { remove_field => ["type", "log", "event"] } mutate { convert => ["@version", "integer"] }}output {# stdout { } elasticsearch { hosts => ["https://your_es_server_ip:9200"] index => "logstash-nginx-%{+YYYY.MM.dd}" user => "elastic" password => "your_es_password" ssl => true # 利用https毗连 ssl_certificate_verification => false # 能否允许去CA机构验证证书能否有效，默认是true cacert => "/logstash/config/http_ca.crt" }}

13、施行如下号令摆设logstash

docker run -d --name logstash --hostname nginx --restart=always -v /root/proxy/logs:/nginx/logs -v /root/logstash/config:/logstash/config docker.elastic.co/logstash/logstash:8.6.1 -f /logstash/config/logstash.conf

留意：需要将Nginx日记目次挂进容器内，按照本身现实情况修改。

14、登录kibana，点击Discover，点击红框所示按钮

15、获取索引数据，如图红框所示：

16、当看见如下界面时，恭喜你搭建胜利。