[CDH] CM5.9.2和CDH5.9.2集群安装

环境:redhat6.5、jdk1.7、python2.7.6
机器:3台

一、配置免密和mysql
1.首先修改/etc/hosts(root,所有节点)

192.168.52.90 CDHM01
192.168.52.101 CDHD01
192.168.52.102 CDHD02

2.所有节点ssh免密(root,主节点)

$ ssh-keygen -t rsa
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ chmod 600 ~/.ssh/authorized_keys

然后scp到其他节点:

$ scp ~/.ssh/authorized_keys root@CDHD01:~/.ssh/
$ scp ~/.ssh/authorized_keys root@CDHD02:~/.ssh/

 

3.安装mysql server并创建数据库
记得设置root的密码:$

mysql -u root -p
mysql>use mysql;
mysql>update user set password=password('root') where user='root';

创建数据库:

#hive
create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
#Hue
create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
#Oozie
create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

#授权root用户在主节点拥有所有数据库的访问权限

mysql>grant all on *.* to root@"%" Identified by "root";
mysql>flush privileges;

4.关闭防火墙

$ service iptables stop
$ chkconfig iptables off

 

5.配置ntp服务,设置主节点作为ntp服务器。这里不做描述

 

二、安装Cloudera Manager
6.安装manager

$ mv cloudera-manager-el6-cm5.9.2_x86_64.tar.gz /opt
$ cd /opt
$ tar zxvf cloudera-manager-el6-cm5.9.2_x86_64.tar.gz

并将parcel包放到cloudera下的parcel-repo目录

$ cp CDH-5.9.2-1.cdh5.9.2.p0.3-el6.parcel* /opt/cloudera/parcel-repo/

从网上下载mysql的JDBC驱动,放到cm-5.9.2对应lib下:

$ cp mysql-connector-java-5.1.40-bin.jar /opt/cm-5.9.2/share/cmf/lib

初始化数据库

</pre>
$ sh /opt/cm-5.9.2/share/cmf/schema/scm_prepare_database.sh mysql cm -h 192.168.52.90 -u root --scm-host 192.168.52.90 scm scm scm

 

三、安装Agent
7.配置主节点和子节点

vi /opt/cm-5.9.2/etc/cloudera-scm-agent/config.ini然后修改
 server_host=CDHM01

然后将cm分发到各子节点

$ scp -r /opt/cm-5.9.2 root@CDHD01:/opt/
 $ scp -r /opt/cm-5.9.2 root@CDHD02:/opt/

 

 

8.创建cloudera-scm用户(所有节点)

$ useradd --system --home=/opt/cm-5.9.2/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

 

 

9.启动Manager和agent
主节点执行:

$ /opt/cm-5.9.2/etc/init.d/cloudera-scm-server start
$ /opt/cm-5.9.2/etc/init.d/cloudera-scm-agent start

子节点执行:

$ /opt/cm-5.9.2/etc/init.d/cloudera-scm-agent start

这时,打开 http://cdhm01:7180即可看到登录界面,默认用户名密码都是admin

 

 

四、安装CDH5集群

10.1登陆后选择 Cloudera Express版本,然后为CDH集群安装指定主机 192.168.52.90,192.168.52.101,192.168.52.102

 

10.2集群安装-选择存储库,选择 5.9.2版本的parcel即可

其余的按照指南安装即可,以下仅对几部做出描述:

版本列表

集群服务选择:

数据库信息填写:

 

 

常见报错及解决
报错1:安装Hive服务,Error: /CDH-5.9.2-1.cdh5.9.2.p0.3/lib/hive/lib/hive-cli-1.1.0-cdh5.9.2.jar org.apache.hive.beeline.HiveSchemaTool -verbose -dbType
mysql -initSchema
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver
at org.apache.hive.beeline.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:79)
…more
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
解决方法:将mysql jdbc的jar包放到所有节点的/opt/cloudera/parcels/CDH-5.9.2-1.cdh5.9.2.p0.3/lib/hive/lib下
scp mysql-connector-java-5.1.40-bin.jar root@cdhd01:/opt/cloudera/parcels/CDH-5.9.2-1.cdh5.9.2.p0.3/lib/hive/lib/

报错2:安装Oozie服务,Error:Error: Could not connect to the database: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
解决方法:将mysql jdbc的jar包放到所有节点的/var/lib/oozie/下

 

报错3:安装Spark Error:JAVA_HOMEis not set and could not be found.
解决方法: 找到/opt/cm-5.9.2/lib64/cmf/service/client/deploy-cc.sh加上如下内容
# attempt to find java
export JAVA_HOME=/usr/local/jdk1.7.0_79
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
报错2:安装Spark服务,Error: JAVA_HOME is not set and could not be found.
解决方法:

分类上一篇:无,已是最新文章    分类下一篇:

Leave a Reply