基于koca的thinjar打包方案

fatjar

对于微服务bootapp,KOCA 默认使用 spring-boot-maven-pligin 插件打包,将 依赖包、业务代码、Tomcat 或其他servlet容器全部打包到一个jar包中,即fatjar,KOCA fatjar打包方案请参考文档:fatjar

fatjar的好处在微服务最佳实践中显而易见:

  • 一个微服务独占整个虚拟机或容器的资源,没必要共享servlet容器,也就没必要将servlet容器单独维护;
  • 保证开发和运行时的servlet容器一致,避免可能出现的版本不兼容问题;
  • 制品传输比较方便,毋需担心依赖丢失等问题。

但是相较于最佳实践,一些相对传统的使用场景却并不希望使用fatjar,比如

  • 希望依赖包可以单独放在外部文件夹,以缩小制品大小;
  • 希望增量升级以减小升级影响范围。

这些场景其实也是在情理之中,特别是在升级过程中,大部分场景下都只需要升级业务代码,依赖包基本不变,因此KOCA提供了一套和fatjar相对应的thinjar解决方案,来适应这些场景。

thinjar

在pom文件中添加插件依赖:

<build>
  <plugins>
    <!-- dependency plugin: 生成maven依赖加载顺序,依赖包外置后,根据此文件中的顺序加载jar包 -->
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-dependency-plugin</artifactId>
      <configuration>
        <outputFile>${build.directory}/thin-classpath.idx</outputFile>
        <includeScope>runtime</includeScope>
      </configuration>
      <executions>
        <execution>
          <id>build-classpath</id>
          <goals>
            <goal>build-classpath</goal>
          </goals>
          <phase>package</phase>
        </execution>
      </executions>
    </plugin>
    <!-- jar plugin: 排除resource目录下的文件,只保留业务代码class,配置文件会单独打包 -->
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-jar-plugin</artifactId>
      <configuration>
        <archive>
          <manifest>
            <addDefaultImplementationEntries>true</addDefaultImplementationEntries>
            <addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
          </manifest>
        </archive>
      </configuration>
      <executions>
        <execution>
          <id>plain-jar</id>
          <phase>package</phase>
          <configuration>
            <excludes>
              <!-- 若有配置文件没有被排除,可以在这里添加规则 -->
              <exclude>config/**</exclude>
              <exclude>*.yml</exclude>
              <exclude>logback-spring.xml</exclude>
              <exclude>banner.txt</exclude>
            </excludes>
            <classifier>bootapp</classifier>
          </configuration>
          <goals>
            <goal>jar</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
    
    <!-- spring boot maven plugin: 依然在repackage阶段使用spring-boot-maven-plugin打包 -->
    <plugin>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-maven-plugin</artifactId>
      <configuration>
        <!-- 排除所有依赖,依赖包会在assembly plugin统一处理 -->
        <includes>
          <include>
            <!-- 随便写一个不存在的依赖包即可 -->
            <groupId>not.exists.groupId</groupId>
            <artifactId>not.exists.artifactId</artifactId>
          </include>
        </includes>
      </configuration>
      <executions>
        <execution>
          <id>default</id>
          <phase>package</phase>
          <configuration>
            <classifier>bootapp</classifier>
          </configuration>
          <goals>
            <goal>build-info</goal>
            <goal>repackage</goal>
          </goals>
        </execution>
      </executions>
      <!-- koca-boot-thin-layout: 扩展spring boot maven plugin,重新制定可执行jar包的Launcher类 -->
      <!-- Launcher类更改为:com.szkingdom.koca.boot.loader.KocaThinJarLauncher -->
      <!-- 支持按照maven依赖顺序加载外部依赖包 -->
      <dependencies>
        <dependency>
          <groupId>szkingdom.yf.koca.base</groupId>
          <artifactId>koca-boot-thin-layout</artifactId>
          <version>${koca-thin-layout-version}</version>
        </dependency>
      </dependencies>
    </plugin>

    <!-- assebly plugin: 完成配置文件、依赖包分组、最终打包等工作 -->
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-assembly-plugin</artifactId>
      <version>3.1.1</version>
      <executions>
        <execution>
          <phase>package</phase>
          <goals>
            <goal>single</goal>
          </goals>
        </execution>
      </executions>
      <configuration>
        <descriptors>
          <!-- assembly descriptor 文件,内容见下文-->
          <descriptor>src/assembly/assembly_thin.xml</descriptor>
        </descriptors>
        <outputDirectory>target</outputDirectory>
      </configuration>
    </plugin>
  </plugins>
</build>

注意:这里一定要指定maven-assembly-plugin插件的版本为3.1.1+,否则会出现依赖包打包不正确的bug

assemly插件用到的的描述文件assembly_thin.xml:

<?xml version='1.0' encoding='UTF-8'?>
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">

    <!-- 打包完成后文件名以“-bin”结尾 -->
    <id>bin</id>
    <formats>
        <!-- 压缩格式为tar.gz -->
        <format>tar.gz</format>
    </formats>
    <includeBaseDirectory>true</includeBaseDirectory>
    <!-- 文件拷贝 -->
    <fileSets>
        <!-- bin目录: 拷贝运维脚本(Windows) -->
        <fileSet>
            <directory>${project.basedir}/src/bin</directory>
            <outputDirectory>bin</outputDirectory>
            <includes>
                <include>*_thin.bat</include>
                <include>shutdown.bat</include>
            </includes>
            <lineEnding>dos</lineEnding>
        </fileSet>
        <!-- bin目录: 拷贝运维脚本(Linux)-->
        <fileSet>
            <directory>${project.basedir}/src/bin</directory>
            <outputDirectory>bin</outputDirectory>
            <includes>
                <include>*_thin.sh</include>
                <include>shutdown.sh</include>
            </includes>
            <lineEnding>unix</lineEnding>
            <fileMode>0755</fileMode>
        </fileSet>
        <!-- conf目录: 从编译目录拷贝配置文件 -->
        <fileSet>
            <directory>${project.build.directory}/classes</directory>
            <outputDirectory>conf</outputDirectory>
            <includes>
                <!-- 若有漏掉的配置文件,可以在这里添加规则 -->
                <include>config/**</include>
                <include>*.yml</include>
                <include>logback-spring.xml</include>
                <include>banner.txt</include>
            </includes>
        </fileSet>
        <!-- lib/bootapp目录: 保存业务jar包,加载jar包时首先从bootapp目录加载 -->
        <fileSet>
            <directory>${project.build.directory}</directory>
            <outputDirectory>lib/bootapp</outputDirectory>
            <includes>
                <include>*-bootapp.jar</include>
            </includes>
        </fileSet>
        <!-- 将classpath idx文件拷贝到lib根目录下 -->
        <fileSet>
            <directory>${project.build.directory}</directory>
            <outputDirectory>lib</outputDirectory>
            <includes>
                <include>thin-classpath.idx</include>
            </includes>
        </fileSet>
    </fileSets>

    <!-- 依赖包拷贝 -->
    <dependencySets>
        <!-- lib/koca目录: 拷贝koca相关的包 -->
        <dependencySet>
            <useProjectArtifact>false</useProjectArtifact>
            <outputDirectory>lib/koca</outputDirectory>
            <outputFileNameMapping>${artifact.artifactId}-${artifact.baseVersion}.${artifact.extension}</outputFileNameMapping>
            <includes>
                <!-- groupId:artifactId,支持通配符 -->
                <include>szkingdom.yf.koca*:*</include>
            </includes>
        </dependencySet>
        <!-- lib/spring目录: 拷贝spring相关的包 -->
        <dependencySet>
            <useProjectArtifact>false</useProjectArtifact>
            <outputDirectory>lib/spring</outputDirectory>
            <outputFileNameMapping>${artifact.artifactId}-${artifact.baseVersion}.${artifact.extension}</outputFileNameMapping>
            <includes>
                <include>org.springframework*:*</include>
            </includes>
        </dependencySet>
        <!-- lib/others目录: 拷贝无法归类的包相关的包 -->
        <dependencySet>
            <useProjectArtifact>false</useProjectArtifact>
            <outputDirectory>lib/others</outputDirectory>
            <outputFileNameMapping>${artifact.artifactId}-${artifact.baseVersion}.${artifact.extension}</outputFileNameMapping>
            <excludes>
                <exclude>org.springframework*:*</exclude>
                <exclude>szkingdom.yf.koca*:*</exclude>
            </excludes>
        </dependencySet>
        <!-- 如果还有自定义的分包规则,可以在这里添加 -->
    </dependencySets>
</assembly>

整个打包流程如下:

  1. 使用maven-dependency-plugin生成依赖包的加载顺序并输出到索引文件,该文件会打包到制品中,用于启动时作为依赖包加载顺序的依据;
  2. 使用maven-jar-plugin在首次生成的jar包中排除配置文件,最后会单独将这些配置文件输出到制品中的指定目录;
  3. 使用spring-boot-maven-plugin将刚刚生成的jar包重新打包,此过程包含重写MANIFEST.MF、禁止将依赖写入jar包内、引入自定义的layout以完成启动时依赖包的加载逻辑,关于自定义layout的部分,有兴趣的可以参考: KOCA启动器
  4. 使用maven-assembly-plugin插件将依赖包按类别输出到不同文件夹,并将分类好的依赖包放入顶层lib目录;将索引文件拷贝至顶层lib目录;将配置文件拷贝到顶层conf目录;将启动、停止、升级脚本拷贝至顶层bin目录,最后把这三个目录压缩打成tar.gz包。

启动脚本startup_thin.sh:

#!/bin/bash
# Author: Chenlei

CMD_DIR=$(cd $(dirname $0); pwd)
cd "$CMD_DIR/.."
CURRENT_DIR=$(pwd)
echo User dir: "$CURRENT_DIR"
CONF_DIR="$CURRENT_DIR/conf"
LIB_DIR="$CURRENT_DIR/lib"
ARGS=$*

# get file name.
FILE_PATH="$LIB_DIR/bootapp"
files=$(ls "$FILE_PATH")
for filename in $files
do
   echo Main Jar: $filename
done
if [[  -f "$CONF_DIR/application.yml" || -f "$CONF_DIR/bootstrap.yml" ]]; then
    echo "Startup app $filename with parameter: $ARGS "
    nohup java \
    -server \
    -Dthin.lib.path="$LIB_DIR" \
    -Dspring.config.location="$CONF_DIR/" \
    -Dkoca.config.location="config" \
    -Dlogging.config="$CONF_DIR/logback-spring.xml" \
    -Dspring.banner.location="file:$CONF_DIR/banner.txt" \
    -Duser.timezone=GMT+08 \
    -jar "$LIB_DIR/bootapp/$filename" \
    $ARGS \
    >./nohup.log 2>&1 &
else
    echo "Configuration folder or files is not exist."
fi

和fatjar区别不大,启动命令添加了-Dthin.lib.path参数,该参数用于指定外部lib目录。

打包过程和启动过程与fatjar无异,需要注意启动脚本的名称从startup.sh/statup.bat变成了startup_thin.sh/startup_thin.bat

增量升级

上面生成的thinjar其实大小和fatjar相差不大,好像并没有瘦身成功,但是我们可以基于这个包完成首次部署,后续升级时就不需要上传整个包了,思路如下:

  1. 首次部署和fatjar差别不大;
  2. 升级时通过脚本对比新旧版本的文件,可以将有差异的文件单独打成增量包;
  3. 上传增量包,使用增量包完成升级,这样可以尽量地减少升级影响的范围。

增量升级的具体文档请参考: thinjar

1 个赞