Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "what are the operation modes of flink". In the operation process of actual cases, many people will encounter such difficulties. Next, let Xiaobian lead you to learn how to deal with these situations! I hope you can read carefully and learn something!

What is Flink?

A framework and distributed processing engine for stateful computation on unbounded and bounded data streams. Flink is designed to run in all common cluster environments, performing computations at memory speeds and at any scale.

download address

Historical download address and latest download address

domestic addresses

operation structure

Compute Architecture: JobManager and TaskManger

JobManager is mainly responsible for scheduling tasks, coordinating checkpoint error recovery, etc.

TaskManger Specific Task Execution

computing framework

Flink SQL, prototype is Ali 2019 open source Blink code.

Flink SQL defines sql statements, and code loading implements data structuring and processing functions.

Table API：

There are source and sink concepts, source is the input source, sink is the output source, the code has related sql methods, no need to define sql statements.

DataStream and DataSet API, pure code to achieve calculation, one responsible for real-time, one service batch processing.

operation mode

information

three modes

Yarn-session mode

Single job model

Divide a portion of resources on Yarn in advance for use by Flink cluster, and share these resources with all tasks submitted by Flink.

Each time a task is submitted, a new flink cluster is created

No hadoop cluster, flink cluster itself.

Local host computing

Mode 1: Local mode

Mode 2: Standalone mode

On Yarn Mode

"What are the operating modes of flink" is introduced here. Thank you for reading. If you want to know more about industry-related knowledge, you can pay attention to the website. Xiaobian will output more high-quality practical articles for everyone!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.