Install the logstash-input-jdbc plug-in

Install the logstash-input-jdbc plug-in to insert data 1. Install gemyum install gem-y

two。 Replace ruby library (two domestic sources + Taobao sources) 2.1.1 with domestic source gem sources-- add https://gems.ruby-china.com/-- remove https://rubygems.org/gem sources-l

2.1.2 modify configuration file cd / opt/inst/logstash722/vi Gemfile# modify Gemfile file # source https://rubygems.org/httpsvi Gemfile.lockgems.ruby-china.com/vi Gemfile.lock#remote: https://rubygems.org/https://gems.ruby-china.com/

2.2 replace ruby as Taobao source (if the above replacement of domestic source is not successful, you can change this method) gem sources-- add https://ruby.taobao.org/-- remove https://rubygems.org/gem sources-l

Since it is not successful to change the ruby image library to Taobao library, you need to modify the data source in the Gemfile file:

Cd / opt/inst/logstash722/vi Gemfile# modify Gemfile file # source https://rubygems.org/source "https://ruby.taobao.org"#remote: https://rubygems.org/remote: https://ruby.taobao.org

3. Install logstash-input-jdbc3.1 to the bin directory of logstash and run the following command to view the available plug-ins cd bin./logstash-plugin list-- verbose

3.2 you can see that the version of logstash-input-jdbc that can be used is 4.3.3. Install JDBCunzip logstash-input-jdbc-4.3.3.zipmv logstash-input-jdbc-4.3.3 / opt/inst/log_jdbc433cd log_jdbc433/

3.3.Configuring Gemfile files vi Gemfilesource 'https://gems.ruby-china.com/'

4. Whether the test was successful 4.1 provide data

A mysql driver package, sql file, and conf configuration file are required

Input {jdbc {# mysql jdbc connection string to our backup databse jdbc_connection_string = > "jdbc:mysql://169.254.211.100:3306/test" # the user we wish to excute our statement as jdbc_user = > "root" jdbc_password = > "root" # the path to our downloaded jdbc driver jdbc_driver_library = > "/ opt/inst/logstash722/sql/mysql-connector-java-5.1.40-bin. Jar "# the name of the driver class for mysql jdbc_driver_class = >" com.mysql.jdbc.Driver "jdbc_paging_enabled = >" true "jdbc_page_size = >" 50000 "statement_filepath = >" / opt/inst/logstash722/sql/logs_20171206.sql "schedule = >" * / 1 * "type = >" jdbc "} filter {json {source = >" Message "remove_field = > [" message "]} output {elasticsearch {hosts = >" 169.254.211.100 codec 9200 "index = >" logs_20171206 "document_id = >"% {id} "} stdout {codec = > json_lines}} 4.2 implement logstash to write files to cd / opt/inst/logstash722/bin./logstash-f / opt/logfiter/mysql.conf in mysq database l

Http://169.254.211.100:5601/app/kibana#/dashboard/1e4f4310-c638-11e9-9ea7-f9ef9537f0f4?_g=(refreshInterval%3A(display%3AOff%2Cpause%3A!f%2Cvalue%3A0)%2Ctime%3A(from%3Anow-1y%2Cmode%3Aquick%2Cto%3Anow))

1566580763148