Kettle+MongoDB data synchronization to MySQL
Kettle+MongoDB data synchronization to MySQL
1. Preface:
The date type in MongoDB is stored as UTC (Coordinated Universal Time), and the isodate type is equal to GMT (Greenwich mean time) time. Beijing is in the + 8 area, so mongo shell will subtract 8 from the current GMT+0800 time and store it as GMT time.
2. Overview of extraction operation
3. Component selection:
4. Incremental processing:
It is correct to query in MongoDB as follows:
> db.xamessages.find ({created_at: {$gte:ISODate ("2016-08-16 18:33:36")}, yy_id:48}) .sort ({created_at:1}) .limit (1)
But the query in kettle mongodb will report an error.
Solution:
{"$query": {"created_at": {"$gte": {"$date": "2016-11-03T00:00:00Z"}
Kettle Wikipedia: http://wiki.pentaho.com/display/BAD/Create+a+Parameterized+Report+with+MongoDB
