iProbe/Docs
0
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

添加 'ELK/fluent收集不同ns的日志.md'

Browse source
This commit is contained in:
iProbe 2023-03-27 18:51:58 +08:00
parent cd1a3cd55b
commit 0be991f285
1 changed files with 67 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

67
ELK/fluent收集不同ns的日志.md Normal file
View file

@ -0,0 +1,67 @@
```yaml
# 区分收集csc,erp,oms等不同命名空间的日志
# 区分收集的原因为：同一个index中字段数据不同会导致冲突，若无冲突可以一起收集
output.conf: |-
# <match **> ## 收集所有日志
# <match kubernetes.var.**_csc_** kubernetes.var.**_erp_** kubernetes.var.**_oms_** kubernetes.var.**_platform_** kubernetes.var.**_srm_**> ## 收集确定的命名空间服务中的日志
<match kubernetes.var.**_csc_**> ## 收集csc命名空间日志
@id elasticsearch-csc ## 与其他配置不同
@type elasticsearch
@log_level info
type_name _doc
include_tag_key true
host elasticsearch-headless
port 9200
logstash_format true ## logstash格式化index名称，默认以logstash开头的index
logstash_prefix csc ## logstash_format开启以后，设置index名称，则index为csc-yyyy.MM.dd
request_timeout 15s
reload_connections false
reconnect_on_error true
reload_on_failure true
suppress_type_name true
<buffer>
@type file
path /var/log/fluentd-buffers/kubernetes.system.csc.buffer ## 与其他配置不同
flush_mode interval
flush_thread_count 2
flush_interval 5s
retry_type exponential_backoff
retry_forever
retry_max_interval 30
chunk_limit_size 10M
queue_limit_length 1
total_limit_size 500M
overflow_action block
</buffer>
</match>
<match **>
@id elasticsearch
@type elasticsearch
@log_level info
type_name _doc
include_tag_key true
host elasticsearch-headless
port 9200
logstash_format true
request_timeout 15s
reload_connections false
reconnect_on_error true
reload_on_failure true
suppress_type_name true
<buffer>
@type file
path /var/log/fluentd-buffers/kubernetes.system.buffer
flush_mode interval
flush_thread_count 2
flush_interval 5s
retry_type exponential_backoff
retry_forever
retry_max_interval 30
chunk_limit_size 10M
queue_limit_length 1
total_limit_size 500M
overflow_action block
</buffer>
</match>
```