[Streaming] Storm jar打包脚本2,从工程bin打包,代替maven/mvn

类似[Streaming] Storm jar打包脚本1,代替maven/mvn

为方便测试,将eclipse export成jar包的过程进一步精简。
其中,工程的bin目录为:/home/hadoop/workspace/stmStorm/bin
使用方法: sh storm_jar_tool2.sh /home/hadoop/shellTest/stmStorm
则会 将bin下的class进行打包==>将常用lib合并打包,生成 工程名称.jar (stmStorm.jar)
需要lib、storm_jar_tool2.sh。其中lib包含需要的所有jar,storm_jar_tool.sh进行打包

1.lib说明。其中lib包含需要的所有jar,这里给出一个自己常用的完整列表
基于版本:JDK-jdk1.7.0_79、storm-1.0.2、flume-1.7.0、Kafka-2.11、Hadoop-hadoop-2.7.3、apache-hive-2.1.1、hbase-1.2.4-bin
lib列表:

aopalliance.jar
aspectjweaver.jar
c3p0-0.9.1.2.jar
commons-compress-1.9.jar
commons-lang-2.3.jar
commons-logging-1.1.1.jar
curator-client-2.5.0.jar
curator-framework-2.5.0.jar
curator-recipes-2.5.0.jar
guava-19.0.jar
jdom.jar
jedis-2.9.0.jar
json-simple-1.1.jar
kafka_2.11-0.10.1.0.jar
kafka-clients-0.10.1.0.jar
kafka-streams-0.10.1.0.jar
kafka-tools-0.10.1.0.jar
log4j.jar
metrics-core-2.2.0.jar
ojdbc14.jar
postgresql-9.3-1102.jdbc4.jar
quartz-1.8.6.jar
scala-compiler.jar
scala-library.jar
slf4j-api-1.6.1.jar
spring-aop-3.2.5.RELEASE.jar
spring-beans-3.2.5.RELEASE.jar
spring-context-3.2.5.RELEASE.jar
spring-context-support-3.2.5.RELEASE.jar
spring-core-3.2.5.RELEASE.jar
spring-expression-3.2.5.RELEASE.jar
spring-jdbc-3.2.5.RELEASE.jar
spring-tx-3.2.5.RELEASE.jar
storm-cassandra-1.0.2.jar
storm-elasticsearch-1.0.2.jar
storm-eventhubs-1.0.2.jar
storm-eventhubs-1.0.2-jar-with-dependencies.jar
storm-hdfs-1.0.2.jar
storm-hive-1.0.2.jar
storm-jdbc-1.0.2.jar
storm-kafka-1.0.2.jar
storm-kafka-client-1.0.2.jar
storm-mongodb-1.0.2.jar
storm-mqtt-1.0.2.jar
storm-redis-1.0.2.jar
storm-solr-1.0.2.jar
storm-sql-kafka-1.0.2.jar
xerces-2.6.2.jar
zookeeper-3.5.1-alpha.jar

 

2.storm_jar_tool.sh说明。storm_jar_tool.sh将lib的所有jar解压、将eclipse打包的jar解压、然后结构化构重新打包

#!/bin/sh

##################################################################
## Function:如果程序执行出错,exit 1(固定的)
##################################################################
issucc()
{
if [ $? -ne 0 ]; then
echo "[`date +'%Y%m%d %H%M%S'`] last command execute failed!!!"
exit 1
fi
}

####################################
## Basic Variables
####################################
#get the shell path
PATH_CURRENT=$(cd `dirname $0`; pwd)
cd ${PATH_CURRENT}

#Create the source package and target package
if [ ! -d "source" ]; then
mkdir source
else
rm -rf source/*
fi
if [ ! -d "target" ]; then
mkdir target
else
rm -rf target/*
fi

####################################
## Step1 jar xvf all the jars
####################################
cp lib/* source/
cp -r ${1}/bin/* source/

cd source
for file in `ls ./*`
do
#echo ${file}
if test -f $file
then
jar xvf $file
fi
done
rm -f *jar

####################################
## Step2 jar cvf the whole package
####################################
jar cvf ${PATH_CURRENT}/target/${1##*/} ./*
issucc

echo "Finished.Path: ${PATH_CURRENT}/target/${1##*/}.jar"

 

3.使用
Step3.1 比如我eclipse工程路径/home/hadoop/workspace/stmStorm,则运行命令:

sh storm_jar_tool2.sh /home/hadoop/workspace/stmStorm

最后会显示Finished.Path: /home/hadoop/shellTest/stmStorm/sbin/target/stmStorm.jar

 

Step3.2 使用storm提交topology

storm jar stmStorm.jar services.basic.WordCountKafkaTopology kafka_spout
分类上一篇:     分类下一篇:

Leave a Reply