How to deploy Flume Interceptors
This article mainly introduces "how to deploy Flume Interceptors". In daily operation, I believe many people have doubts about how to deploy Flume Interceptors. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the doubts about "how to deploy Flume Interceptors". Next, please follow the editor to study!
1. Timestamp Interceptor
In event headers, add the timestamp tag
[root@hftest0001 conf] # pwd/opt/apache-flume-1.6.0-bin/conf [root@hftest0001 conf] # vi s-exec_c-m_s-logger.conf agent.sources = exec_tailagent.channels = memoryChannelagent.sinks = loggerSinkagent.sources.exec_tail.type = execagent.sources.exec_tail.command = tail-F / opt/flume-data/exec-tail.logagent.sources.exec_tail.channels = memoryChannelagent.sources.exec_tail.interceptors = i1agent.sources.exec_tail .promotors.i1.type = timestamp#agent.sources.exec_tail.interceptors.i1.preserveExisting = trueagent.sinks.loggerSink.type = logger#agent.sinks.loggerSink.maxBytesToLog = 1agent.sinks.loggerSink.channel = memoryChannelagent.channels.memoryChannel.type = memoryagent.channels.memoryChannel.capacity = 100 examples = > result Event: {headers: {timestamp=1452668731658} body: 78 78 xx}
II. Host Interceptor
Get the ip or hostname of the server deployed by agent, or add it to the host tag
[root@hftest0001 conf] # pwd/opt/apache-flume-1.6.0-bin/conf [root@hftest0001 conf] # vi s-exec_c-m_s-logger.conf agent.sources.exec_tail.type = execagent.sources.exec_tail.command = tail-F / opt/flume-data/exec-tail.logagent.sources.exec_tail.channels = memoryChannelagent.sources.exec_tail.interceptors = i1 i2agent.sources.exec_tail.interceptors.i1.type = timestamp#agent.sources. Exec_tail.interceptors.i1.preserveExisting = trueagent.sources.exec_tail.interceptors.i2.type = host#agent.sources.exec_tail.interceptors.i2.userIP = false = > default:true is set to false The value obtained is hostname#agent.sources.exec_tail.interceptors.i2.hostHeader = hostname = > key in default:header is host, and you can set the change # agent.sources.exec_tail.interceptors.i2.preserveExisting = trueagent.sinks.loggerSink.type = logger#agent.sinks.loggerSink.maxBytesToLog = memoryChannelagent.channels.memoryChannel.type = memoryChannelagent.channels.memoryChannel.type = 100 cycles = > result Event: {headers: {timestamp=1452669139164 Host=$ {ip}} body: 78 78 xx}
III. Static Interceptor
Customize to add events to header: purpose, if you want to know which dc, that country, and that city. Data can also be diverted according to these header information, and so on.
[root@hftest0001 conf] # pwd/opt/apache-flume-1.6.0-bin/conf [root@hftest0001 conf] # vi s-exec_c-m_s-logger.conf # agent.sources.exec_tail.interceptors.i2.userIP = false#agent.sources.exec_tail.interceptors.i2.hostHeader = hostname#agent.sources.exec_tail.interceptors.i2.preserveExisting = trueagent.sources.exec_tail.interceptors.i3.type = staticagent.sources.exec_tail.interceptors.i3.key = Countryagent.sources.exec_tail.interceptors.i3.value = USAagent.sources.exec_tail.interceptors.i4.type = staticagent.sources.exec_tail.interceptors.i4.key = cityagent.sources.exec_tail.interceptors.i4.value = NEW_YORKagent.sinks.loggerSink.type = logger#agent.sinks.loggerSink.maxBytesToLog = 1agent.sinks.loggerSink.channel = memoryChannelagent.channels.memoryChannel.type = memoryagent.channels.memoryChannel.capacity = 100 stories = > resultEvent: {headers: {timestamp=1452669538967 Host=$ {IP}, city=NEW_YORK, country=USA} body: 78 78 xx} so far The study on "how to deploy Flume Interceptors" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!
