Shulou(Shulou.com)06/03 Report--

1. Introduction of ELK

ELKStacks is a combination of technology stacks, namely Elasticsearch, Logstash, and Kibana.

ELK Stack:

1. Scalability: designed with highly scalable distributed architecture, which can support daily TB-level data

2. Easy to use: various statistics and visualization of log data can be made through the graphic page.

3. High query efficiency: can achieve second-level data acquisition, processing and search

Https://www.elastic.co/cn/products/elasticsearch

Https://www.elastic.co/cn/products/kibana

Https://www.elastic.co/cn/products/beats/filebeat

Https://www.elastic.co/cn/products/beats/metricbeat

II. ELK architecture

Logstash: an open source server-side data processing pipeline that can collect data from multiple sources at the same time, transform data, and then store the data in a database.

Elasticsearch: search, analyze, and store data.

Kibana: data visualization.

Beats: a platform for lightweight collectors that send data from edge machines to Logstash and Elasticsearch.

Filebeat: lightweight log collector.

Https://www.elastic.co/cn/

Https://www.elastic.co/subscriptions

Input: input and output data can be Stdin, File, TCP, Redis, Syslog, etc.

Filter: filter and format the log. There are rich filtering plug-ins: Grok regular capture, Date time processing, Json codec, Mutate data modification and so on.

Output: output. The output target can be Stdout, File, TCP, Redis, ES, etc.

III. Basic concepts of ElasticSearch3.1

Node: a server running a single instance of ES

Cluster: one or more nodes form a cluster

Index: an index is a collection of multiple documents

Each record in Document:Index is called Document, and several documents build an Index.

Type: an Index can define one or more types to group Document logically

Minimum unit of Field:ES storage

Shards:ES divides the Index into several parts, each of which is a slice

One or more copies of Replicas:Index

ES

Relational database (such as Mysql)

Index

Database

Type

Table

Document

Row

Field

Column

3.2 Cluster deployment

First of all, do the initialization configuration of the system and install jdk

# 1) System initialization on each Serverscat > > / etc/security/limits.conf / etc/sysctl.conf / etc/elasticsearch/elasticsearch.yml

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.