Get the App
SLTechnology News&Howtos  ›  Internet Technology  › 

Summary of problems encountered in sqoop2 1.99.6 and source code modification

Shulou Source: shulou.com Published: 2022-06-03 07:33:34 11月23日 Update

1. When the cardinality of PartitionColumn is 1 (as shown in the following figure). Will report an error.

The error is as shown in the following picture

The source code is as follows

\ sqoop-1.99.6-bin-hadoop200\ connector\ connector-generic-jdbc\ src\ main\ java\ org\ apache\ sqoop\ connector\ jdbc\ GenericJdbcPartitioner.java

Change the source code to

Can

The parameter Null value allowed for the partition column: false in 2.Job will not report an error even if partition column has null, so the record for null is imported into HDFS.

Conclusion: it is not a problem. When this parameter is false, the map of pid is null will not be de-derived, but only when it is true, so it is correct not to report an error.

3. The problem of date and time. The specific modification method is as follows.

E:\ IdeaProject\ sqoop-1.99.6-bin-hadoop200\ connector\ connector-generic-jdbc\ src\ main\ java\ org\ apache\ sqoop\ connector\ jdbc\ GenericJdbcExtractor.java

The time type in 4.oracle will report an error when doing pid.

If (JDBC_DRIVER_ORACLE.equals (jdbcDriver)) {

/ / if it is oracle, do special treatment.

Conditions.append ("to_timestamp (\'") .append (sdf.format ((java.util.Date) lowerBound)) .append ("\', 'yyyy-mm-dd hh34:mi:ss.ff')")

Conditions.append ("

Tags: Time report problem special metric parameter source code cardinality character string method date format type conclusion or processing source code Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno macOS Docker Redmi vpn Shulou Tech Info