使用idea阅读Kafka源码

本文涉及到的环境：

操作系统：Windows 7
Idea 版本：IntelliJ IDEA 2016.3.4 Build #IU-163.12024.16, built on January 31, 2017
Kafka 版本：Kafka 0.8.2.0
Gradle 版本：gradle-4.0.1
JDK 版本：jdk1.7.0
Scala 版本：2.10.4

首先到http://archive.apache.org/dist/kafka/里面下载你需要的Kafka源码，本文选自的是kafka-0.8.2.0。

因为Kafka代码自0.8.x之后就使用 Gradle 来进行编译和构建了，所以如果直接通过 idea 来打开上面下载的Kafka 源码会出现很多错误。我们需要先安装好 Gradle。

到 https://gradle.org/releases/ 下载你需要的 Gradle，本文选择的是 gradle-4.0.1。下载并解压 Gradle 到一个目录，然后创建一个环境变量 GRADLE_HOME 指向解压的目录，再将 %GRADLE_HOME%\bin 加到 PATH 环境变量中，保存并打开 cmd 窗口，在里面输入 gradle -v，如果输出类似于下面的信息，说明 Gradle 已经加入到环境变量中了。

$ gradle -v

------------------------------------------------------------
Gradle 4.0.1
------------------------------------------------------------

Build time:   2017-07-07 14:02:41 UTC
Revision:     38e5dc0f772daecca1d2681885d3d85414eb6826

Groovy:       2.4.11
Ant:          Apache Ant(TM) version 1.9.6 compiled on June 29 2015
JVM:          1.7.0 (Oracle Corporation 21.0-b17)
OS:           Windows 7 6.1 amd64

当然，将 Gradle 加入到环境变量是可选的，只是可以直接在命令行里面使用 gradle。

安装完 Gradle 之后，我们到Kafka的源码目录下，然后在 cmd 窗口执行下面的命令：

$ gradle idea
Starting a Gradle Daemon, 2 busy and 1 incompatible Daemons could not be reused, use --status for details
Download https://repo1.maven.org/maven2//nl/javadude/gradle/plugins/license-gradle-plugin/0.10.0/license-gradle-plugin-0.10.0.pom
Download https://repo1.maven.org/maven2//com/google/guava/guava/17.0/guava-17.0.pom
Download https://repo1.maven.org/maven2//com/mycila/maven-license-plugin/maven-license-plugin/1.10.b1/maven-license-plugin-1.10.b1.pom
Download https://repo1.maven.org/maven2//com/google/guava/guava-parent/17.0/guava-parent-17.0.pom
Download https://repo1.maven.org/maven2//org/sonatype/oss/oss-parent/7/oss-parent-7.pom
Download https://repo1.maven.org/maven2//com/mycila/parent-pom/5/parent-pom-5.pom
Download https://repo1.maven.org/maven2//com/mycila/xmltool/xmltool/3.3/xmltool-3.3.pom
Download https://repo1.maven.org/maven2//org/codehaus/plexus/plexus-utils/2.0.5/plexus-utils-2.0.5.pom
Download https://repo1.maven.org/maven2//org/codehaus/plexus/plexus/2.0.6/plexus-2.0.6.pom
Download https://repo1.maven.org/maven2//com/mycila/maven-license-plugin/maven-license-plugin/1.10.b1/maven-license-plugin-1.10.b1.jar
Download https://repo1.maven.org/maven2//com/google/guava/guava/17.0/guava-17.0.jar
Download https://repo1.maven.org/maven2//com/mycila/xmltool/xmltool/3.3/xmltool-3.3.jar
Download https://repo1.maven.org/maven2//org/codehaus/plexus/plexus-utils/2.0.5/plexus-utils-2.0.5.jar
Download https://repo1.maven.org/maven2//nl/javadude/gradle/plugins/license-gradle-plugin/0.10.0/license-gradle-plugin-0.10.0.jar
Building project 'core' with Scala version 2.10.4
:ideaModule
:ideaProject

.........................................................................
这里省了一堆log
.........................................................................
:contrib:hadoop-producer:idea

BUILD SUCCESSFUL in 1m 7s
9 actionable tasks: 9 executed

gradle 会自动下载好 Kafka 项目依赖的jar包。

1、你在执行 gradle idea 命令的时候，可能会遇到以下的错误：

$ gradle idea
Starting a Gradle Daemon, 1 incompatible Daemon could not be reused, use --status for details
Building project 'core' with Scala version 2.10.4

FAILURE: Build failed with an exception.

* Where:
Build file 'D:\user\iteblog\kafka-0.8.2\build.gradle' line: 131

* What went wrong:
A problem occurred evaluating root project 'kafka-0.8.2'.
> Failed to apply plugin [class 'org.gradle.api.plugins.scala.ScalaBasePlugin']
   > No such property: useAnt for class: org.gradle.api.tasks.scala.ScalaCompileOptions

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED in 9s

我们只需要在 Kafka 源码的 build.gradle 文件里面最顶端添加以下的内容保存即可：

ScalaCompileOptions.metaClass.daemonServer = true
ScalaCompileOptions.metaClass.fork = true
ScalaCompileOptions.metaClass.useAnt = false
ScalaCompileOptions.metaClass.useCompileDaemon = false

2、gradle 默认使用的是 https://repo1.maven.org/maven2/ 源进行下载相关的 jar 依赖，这可能会很慢。我们可以将其修改为阿里的 Maven源，只需要将 build.gradle 文件里面的下面代码：

buildscript {
  repositories {
    mavenCentral()
  }
  apply from: file('gradle/buildscript.gradle'), to: buildscript
}

def slf4jlog4j='org.slf4j:slf4j-log4j12:1.7.6'

allprojects {
  apply plugin: 'idea'
  repositories {
    mavenCentral()
  }
}

修改成

buildscript {
  repositories {
    maven { url 'http://maven.aliyun.com/nexus/content/groups/public/' }
  }
  apply from: file('gradle/buildscript.gradle'), to: buildscript
}

def slf4jlog4j='org.slf4j:slf4j-log4j12:1.7.6'

allprojects {
  apply plugin: 'idea'
  repositories {
    maven { url 'http://maven.aliyun.com/nexus/content/groups/public/' }
  }
}

其他国内比较快的 Maven 仓库源可以参见：《推荐几个国内访问比较快的Maven仓库镜像》。

运行完 gradle idea 命令之后，我们就可以通过 idea 打开 Kafka 源码了，已经不再有错误，如下：