Detailed explanation of the steps for Flume to listen for the increment of oracle table

Requirements: get the incremental information of oracle table, send it to udp514 port, support ip configuration

Steps:

(1) odbc5.jar of the required jar oracle (look under the oracle installation directory / jdbc/lib)

(2) the latest small version of flume's open source package flume-ng-sql-source-1.4.3.jar seems to be 1.5. I can't remember it. This download address is available directly on csdn.

Both jar are copied to flume's lib.

(3) flume configuration file

A1.sources = r1a1.sinks = k1a1.channels = syslog receive configuration # a1.sources.r1.type = syslogudp#a1.sources.r1.port = 514#a1.sources.r1.host = 0.0.0.0#sql source#a1.sources.r1.type = org.keedio.flume.source.SQLSourcea1.sources.r1.hibernate.connection.url = jdbc:oracle:thin:@ 10.9.224.36:1521/soc# Hibernate Database connection propertiesa1.sources.r1.hibernate.connection.user = soc_appa1.sources.r1.hibernate.connection.password = soc_appa1.sources.r1.hibernate.connection.autocommit = truea1.sources.r1.hibernate.dialect = org.hibernate.dialect.Oracle10gDialecta1.sources.r1.hibernate.connection.driver_class = oracle.jdbc.driver.OracleDrivera1.sources.r1.run.query.delay=1a1.sources.r1.status.file.path = / usr/local/flumea1. Sources.r1.status.file.name = a1.sqlSource.status# Custom querya1.sources.r1.start.from = '2017-07-31 07:06:20'a1.sources.r1.custom.query = SELECT CHR (39) | | TO_CHAR (CREATETIME) 'YYYY-MM-DD HH24:MI:SS') | | CHR (39), MSG FROM FLUME_NG_SQL_SOURCE WHERE CREATETIME > TO_DATE ($@ $) 'YYYY-MM-DD HH24:MI:SS') ORDER BY CREATETIME ASCa1.sources.r1.batch.size = 6000a1.sources.r1.max.rows = 1000a1.sources.r1.hibernate.connection.provider_class = org.hibernate.connection.C3P0ConnectionProvidera1.sources.r1.hibernate.c3p0.min_size=1a1.sources.r1.hibernate.c3p0.max_size=10#sql source# Custom Type a1.sinks .k1.type = com.neusoft.utils.SyslogSink# configuration sends the ipa1.sinks.k1.destination = 10.9.224.36a1.channels.c1.type = memorya1.channels.c1.capacity = 10000a1.channels.c1.transactionCapacity = 10000a1.sources.r1.channels = c1a1.sinks.k1.channel = flume-ng agent startup command. / flume-ng agent-c.. / conf-f.. / conf/flume-sql.conf-n A1-Dflume.root.logger=INFO Console &

Since the self-defined sink is also mentioned in my blog, I configured a custom sink.

Summary

The above is the detailed explanation of the incremental steps of Flume monitoring oracle table introduced by the editor. I hope it will be helpful to you. If you have any questions, please leave me a message and the editor will reply to you in time. Thank you very much for your support to the website!