Shulou(Shulou.com)06/02 Report--

This article will explain in detail how to solve the Debezium pit, the content of the article is of high quality, so the editor will share it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

There's something wrong with Snapshot Reader. We are testing that Snapshot is OK in the test environment.

However, when the amount of data becomes larger, and the amount of data becomes more, Snapshot Reader does not achieve the desired results.

Therefore, we changed the synchronization scheme to three steps:

1. Enable schema_only synchronization

2. 5 minutes later, Dump the database to a temporary library

3. Open java multithreading for data consumption, and use mysql jdbc streaming method to write the data above the temporary library to kudu.

4. Open the kudu Sink of Kafka-Connect, and let the new kafka add, delete and modify data to overwrite the original data.

The current use is stable, continue to observe.

On how to solve the Debezium pit to share here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.