1、概述
本节演示如何在本地编译、运行 Flink 源码。
2、版本说明
| 名称 | 版本 |
|---|---|
| flink | 1.13.6 |
| jdk | 1.8 |
| Maven | 3.2.5 |
| 操作系统 | Mac |
3、编译 Flink 源码
1）从 github 下载 Flink 源码
git clone https://github.com/apache/flink
2）切换到目标分支
git checkout remotes/origin/release-1.13
3）编译源码
mvn clean install -DskipTests -Dfast -Dcheckstyle.skip=true
问题：出现某些依赖找不到
解决：单独下载问题 Jar 包，并 install 到本地仓库
问题：依赖下载慢、依赖下载失败
解决：vpn
4）IDEA启动 Standalone 集群
1.配置准备
a）在 flink-dist 模块下 bin 同级目录 创建 log 目录、lib 目录
b）把 resource 目录中的 flink-conf.yaml 文件拷贝到 conf 目录
c）把 target 里的flink-dist.jar包放到 lib 目录，还有四个 log 相关的 jar 包和 zookeeper 相关的 jar 包
file
2.启动 JobManager
启动类：
flink-runtime 模块下
org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint#main
IDEA 启动参数配置：
VM参数：
-Xmx1073741824
-Xms1073741824
-XX:MaxMetaspaceSize=268435456
-Dlog.file=/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/log/flink-standalonesession.log
-Dlog4j.configuration=file:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/conf/log4j-console.properties
-Dlog4j.configurationFile=file:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/conf/log4j-console.properties
-Dlogback.configurationFile=file:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/conf/logback-console.xml
-classpath
/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/flink-csv-1.13.6.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/flink-json-1.13.6.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/flink-shaded-zookeeper-3.4.14.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/flink-table-blink_2.11-1.13.6.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/flink-table_2.11-1.13.6.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/log4j-1.2-api-2.17.1.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/log4j-api-2.17.1.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/log4j-core-2.17.1.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/log4j-slf4j-impl-2.17.1.jar:/Users/***/Desktop/flink/flink-dist/target/flink-dist_2.11-1.13.6.jar:::
运行参数：
--configDir /Users/***/Desktop/flink/flink-dist/src/main/flink-bin/conf
--executionMode cluster
-D jobmanager.memory.off-heap.size=134217728b
-D jobmanager.memory.jvm-overhead.min=201326592b
-D jobmanager.memory.jvm-metaspace.size=268435456b
-D jobmanager.memory.heap.size=1073741824b
-D jobmanager.memory.jvm-overhead.max=201326592b
3.启动 TaskManager
启动类：
flink-runtime 模块下
org.apache.flink.runtime.taskexecutor.TaskManagerRunner#main
IDEA 启动参数配置：
VM 参数：
-XX:+UseG1GC
-Xmx536870902
-Xms536870902
-XX:MaxDirectMemorySize=268435458
-XX:MaxMetaspaceSize=268435456
-Dlog.file=/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/log/flink-taskexecutor.log
-Dlog4j.configuration=file:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/conf/log4j-console.properties
-Dlog4j.configurationFile=file:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/conf/log4j-console.properties
-Dlogback.configurationFile=file:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/conf/logback-console.xml
-classpath
/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/flink-csv-1.13.6.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/flink-json-1.13.6.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/flink-shaded-zookeeper-3.4.14.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/flink-table-blink_2.11-1.13.6.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/flink-table_2.11-1.13.6.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/log4j-1.2-api-2.17.1.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/log4j-api-2.17.1.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/log4j-core-2.17.1.jar:/Users/***/Desktop/flink/flink-dist/src/main/flink-bin/lib/log4j-slf4j-impl-2.17.1.jar:/Users/***/Desktop/flink/flink-dist/target/flink-dist_2.11-1.13.6.jar:::
运行参数：
--configDir /Users/***/Desktop/flink/flink-dist/src/main/flink-bin/conf
-D taskmanager.memory.network.min=134217730b
-D taskmanager.cpu.cores=1.0
-D taskmanager.memory.task.off-heap.size=0b
-D taskmanager.memory.jvm-metaspace.size=268435456b
-D external-resources=none
-D taskmanager.memory.jvm-overhead.min=201326592b
-D taskmanager.memory.framework.off-heap.size=134217728b
-D taskmanager.memory.network.max=134217730b
-D taskmanager.memory.framework.heap.size=134217728b
-D taskmanager.memory.managed.size=536870920b
-D taskmanager.memory.task.heap.size=402653174b
-D taskmanager.numberOfTaskSlots=1
-D taskmanager.memory.jvm-overhead.max=201326592b
问题：运行代码后，打印日志控制台无输出
解决：增加控制台日志配置，在 flink 项目中有 log4j-console.properties 和 logback-console.xml，在 IDEA 启动参数中传入日志配置
-Dlog.file=/Users/*/Desktop/flink-1.13.6/log/flink-standalonesession.log
-Dlog4j.configuration=file:/Users/*/Desktop/flink-1.13.6/conf/log4j-console.properties
-Dlog4j.configurationFile=file:/Users/*/Desktop/flink-1.13.6/conf/log4j-console.properties
-Dlogback.configurationFile=file:/Users/*/Desktop/flink-1.13.6/conf/logback-console.xml
5）打开 Flink WebUI 进行验证
localhost:8081
file
4、修改源码
1）打包源码
问题：打包源码时，出现某个 Module 找不到
解决：优先打包报错的 Module，并 install 到本地仓库即可
2）修改源码后，重新打包，并运行验证
问题：修改源码后运行无效果，如增加标准输出不显示
解决：每次修改需要重新打包，因为我们运行的是flink-dist模块下的flink-dist_2.11-1.13.6.jar
file
本文由博客一文多发平台 OpenWrite 发布！