What are the core concepts of flume NG

This article mainly shows you "what are the core concepts of flume NG", which is easy to understand and clear. I hope it can help you solve your doubts. Let the editor lead you to study and learn this article "what are the core concepts of flume NG?"

Core concepts of flume NG

Event

Client

Agent

Source

Channel

Sink

A.Event is the basic unit of Flume data transmission.

B.Flume transmits data from the source to the final destination (such as HDFS) in the form of events.

The c.Event may consist of an optional header and a byte array containing data.

D.Client is a process that wraps the original log into Events and sends them to one or more Agent entities.

E.Client is not required in the topology of Flume, and its purpose is to decouple Flume from the data source system.

f. An Agent contains Source, Channel, Sink, and other components.

G.Agent uses these components to transfer Events from one node to another or to the ultimate destination (file system: such as HDFS).

H.Agent is the most important part of the Flume stream.

I.Flume provides configuration, life cycle management, and monitoring support for these components.

These are all the contents of the article "what are the Core Concepts of flume NG". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!