[Spark] 使用Hive on Spark的Java示例

在使用conn进行SQL操作之前,先执行SQL将hive.exection.engine设置为spark即可使用Hive on Spark。代码示例:


	/**
	 * Spark PreparedStated
	 */
	private static PreparedStatement getPreparedStatement(PreparedStatement statement){
		statement = null;
		try{
			statement = conn.prepareStatement("set hive.execution.engine=spark");
			statement.executeUpdate();
		}catch (Exception e){
			LOGGER.warn("[E_WARN]", e);
		}
		return statement;

	}



	/**
	 * Hive Non Reuturn's update
	 */
	public static int executeNonUpdate(String sqlStr) throws Exception{
		PreparedStatement statement = null;
		statement = getPreparedStatement(statement);
		int i = 0;
		try {
			statement = conn.prepareStatement(sqlStr);
			i = statement.executeUpdate();
		} catch (SQLException e) {
			throw(e);
		} finally {
			if (null != statement) {
				try {
					statement.close();
				} catch (SQLException e) {
					throw(e);
				}
			}
		}
		return i;
	}


实测,在多层join等情况下确实快了数倍

分类上一篇:     分类下一篇:

Leave a Reply