Shulou(Shulou.com)06/01 Report--

This article mainly explains "the method that sqoop uses mysql as metastore". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's way of thinking to study and learn "sqoop's method of using mysql as metastore".

Sqoop.metastore.client.enable.autoconnect

False

If true, Sqoop will connect to a local metastore

For job management when no other metastore arguments are

Provided.

Sqoop.metastore.client.autoconnect.url

Jdbc:mysql://:/sqoop?createDatabaseIfNotExist=true

Sqoop.metastore.client.autoconnect.username

Sqoop

Sqoop.metastore.client.autoconnect.password

Sqoop

CREATE TABLE SQOOP_ROOT (

Version INT

Propname VARCHAR (128) NOT NULL

Propval VARCHAR (256)

CONSTRAINT SQOOP_ROOT_unq UNIQUE (version, propname)

);

-Inserted the following row (* * this did the trick)

INSERT INTO

SQOOP_ROOT

VALUES (

NULL

'sqoop.hsqldb.job.storage.version'

'0'

);

# incremental Import

Sqoop job-- create job3-- meta-connect 'jdbc:mysql://172.18.1.1:3306/sqoop?user=root&password=test'-- import-- connect' jdbc:mysql://172.18.1.1:3306/test'-- username 'root'-- password' test'-- table 'test1'-- columns' id,name,age'-- hive-import-- hive-table default.jnincr1-- incremental append-- check-column 'id'-- last-value 0

Sqoop job-meta-connect 'jdbc:mysql://172.18.1.1:3306/sqoop?user=root&password=test'-list

Sqoop job-- meta-connect 'jdbc:mysql://172.18.1.1:3306/sqoop?user=root&password=test'-- exec job3-username root-- password test-- append

Sqoop job-- create job3-- meta-connect 'jdbc:hsqldb:hsql://172.18.1.1:16000/sqoop'-- import-- connect' jdbc:mysql://172.18.1.1:3306/test'-- username 'root'-- password' 123456'--table 'test_001'-- columns' id,name'-- hive-import-- hive-table default.a3-- incremental append-- check-column 'id'-- last-value 0

Sqoop job-- meta-connect 'jdbc:hsqldb:hsql://172.18.1.1:16000/sqoop'-- exec job3-verbose

This method needs to modify the sqoop source code to solve the problem of transaction locking table.

Thank you for reading, the above is the content of "sqoop uses mysql as a metastore method". After the study of this article, I believe you have a deeper understanding of the problem of sqoop using mysql as a metastore method, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.