How to build Development Environment and data in Flink
This article will explain in detail how to build the development environment and data in Flink. The content of the article is of high quality, so the editor will share it with you for reference. I hope you will have some understanding of the relevant knowledge after reading this article.
Sample environment
Java.version: 1.8.xflink.version: 1.11.1
Create a project
Create a project in idea. Project name: flink-examples
Pom.xml
4.0.0 com.flink.examples flink-examples pom 1.0-SNAPSHOT stream connectors UTF-8 UTF-8 1.8 1.11.1 8 org.apache.flink flink-streaming-java_2.11 ${flink.version} Org.apache.flink flink-clients_2.11 ${flink.version} org.apache.flink flink-table-api-java-bridge_2.11 ${flink.version} org.apache.flink flink-table-planner-blink_2.11 ${flink.version} org.apache.flink flink-table-planner_2.11 ${flink.version} org.apache.flink flink-json ${flink.version} com.google.code. Gson gson 2.8.6 org.apache.logging.log4j log4j-core 2.11.1 org.apache.maven.plugins maven-compiler-plugin UTF-8 ${compiler.version} ${compiler.version} org.apache.maven.plugins maven-shade-plugin package Shade
The project is divided into two modules.
Flink-examples |-connectors (sample module for inter-component connectors) |-examples (simulating e-commerce order data and pushing it to kafka, and flink core data stream processing client) |-stream (data flow and operator, method, window and other sample modules) |-tableapi (example module between table&sql and middleware) |-web (get the storage result calculated by flink operator Provided to the front-end display module)
Connectors module
Pom.xml
Com.flink.examples flink-examples 1.0-SNAPSHOT 4.0.0 connectors org.apache.flink flink-jdbc_2.11 1.10.1 mysql mysql-connector-java 5.1.47 Org.apache.flink flink-connector-kafka_2.11 ${flink.version} org.apache.bahir flink-connector-redis_2.11 1.0 org.apache.flink flink-connector-rabbitmq_2.11 ${flink.version} org.apache.flink flink-connector-elasticsearch7_2.11 ${flink.version} org.apache.flink flink-sql-connector-elasticsearch7_2.11 ${flink.version}
Stream module
Pom.xml
Flink-examples com.flink.examples 1.0-SNAPSHOT 4.0.0 stream org.apache.flink flink-table-common ${flink.version}
Project global sample diagram
Sample data class
The data demonstrated by all subsequent operators come from this class to provide public data.
Import org.apache.flink.api.java.tuple.Tuple3;import java.util.Arrays;import java.util.List / * * @ Description Public sample data * / public class DataSource {/ * sample data collection * Tuple3 is an entity class with fixed three attribute variables F0 return f1jue f2 is used to denote three structural transmission participating variables * @ return * / public static List getTuple3ToList () {/ / Tuple3 = Tuple3 return Arrays.asList (new Tuple3 ("Zhang San", "man", 20), new Tuple3 ("Li Si", "girl", 24), new Tuple3 ("Wang Wu", "man", 29). New Tuple3 (Liu Liu, girl, 32), new Tuple3 (Wu Qi, girl, 18), new Tuple3 (Wu Ba, man, 30)) }} on how to build a development environment in Flink and data sharing here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.
