[HDP] HDP集群环境搭建:2.HDP 2.4.0安装

1.创建httpd服务器,并将相关的hdp包放在目录下 下载地址:
我的是: http://192.168.111.18:8000/hdp
hdp包如下:
HDP-2.4.0.0-centos6-rpm.tar.gz
HDP-UTILS-1.1.0.20-centos6.tar.gz
ambari-2.2.1.0-centos6.tar.gz
hdp.repo

记得在这4个文件的目录执行命令以生成rpm包的依赖关系,否则会报错 repodata/repomd.xml 没有找到那个文件或目录

$ createrepo ./

 

2.配置repo源
找到/var/lib/ambari-server/resources/stacks/HDP/2.4/repos/repoinfo.xml并修改对应操作系统比如redhat6的配置:

  
<os family="redhat6">
    <repo>
      <baseurl>http://192.168.111.18:8000/hdp</baseurl>
      <repoid>HDP-2.4</repoid>
      <reponame>HDP</reponame>
    </repo>
    <repo>
      <baseurl>http://192.168.111.18:8000/hdp</baseurl>
      <repoid>HDP-UTILS-1.1.0.20</repoid>
      <reponame>HDP-UTILS</reponame>
    </repo>
  </os>

修改/etc/yum.repo/ambari.repo,添加如下内容:

[Updates-ambari-2.2.1.0]
name=ambari-2.2.1.0 - Updates
baseurl=http://192.168.111.18:8000/hdp/
gpgcheck=1
gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

[ambari-2.2.1.0]
name=Ambari 2.2.1.0
baseurl=http://192.168.111.18:8000/hdp/
gpgcheck=0
enabled=1

[HDP-UTILS-1.1.0.20]
name=Hortonworks Data Platform Utils Version - HDP-UTILS-1.1.0.20
baseurl=http://192.168.111.18:8000/hdp/
gpgcheck=0
enabled=1

 

3.配置ntp,这里不做详细描述了。(Best,not Must)

 

4.安装HDP

Step1.Get Started
Step2.Select Stack

Step3.Install Options
Step4.Confirm Hosts


Step5.Choose Services.仅仅是测试,选择自己需要的(我除了Falcon都选了),实际使用请考虑架构设计


Step6.Assign Masters.仅仅是测试,所以随即分配,实际使用请务必考虑高可用

Step7.Assign Slaves and Clients

Step8.Customize Services。这里需要补充Hive、Oozie、Accumulo、Knox、SmartSense的
Hive: hive/hive
Oozie:oozie/opozie:
Knox: hddb01 /knox
SmartSense:
–account:smart
–senseID:smart
email:smart@ccb.com
然后下一步
注意:这里有Warning Derby is not recommended for production use. With Derby, Oozie Server HA and concurrent connection support will not
be available. 生产请不要使用

Step9.Review 简单看下,务必保存,然后Deploy。等待进度条即可

Step10.Install,Start and Test自动进行
Step11.Summary

然后CDH集群即安装完成

 

 

 

常见问题及解决:
1.Confirm Hosts的过程中,部分节点出现ERROR 2017-05-10 14:29:21,615 NetUtil.py:85 – SSLError: Failed to connect. Please check openssl
library versions.
WARNING 2017-05-10 14:29:21,616 NetUtil.py:112 – Server at https://HMDB01:8440 is not reachable, sleeping for 10 seconds
解决方法:openssl版本太低或者不全,修复如下:
$ rpm -qa |grep openssl
openssl-1.0.1e-15.el6.x86_64
发现openssl的版本过低,upgrade更新即可,install openssl*更彻底
$ yum install openssl*
然后回到Ambari页面单击Retry

 

2.在Install,Smart and Clients过程中,Accumulo、app timeline server报错 no module named yum
解决方法:问题节点Python版本不对。请在最初安装的时候就使用python2.6.6且带yum、rpm等全的module

 

 

附:我的Step9 Review

Admin Name : admin

Cluster Name : HDP01

Total Hosts : 3 (3 new)

Repositories:

debian7 (HDP-2.4):
http://public-repo-1.hortonworks.com/HDP/debian7/2.x/updates/2.4.0.0

debian7 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/debian6

redhat6 (HDP-2.4):
http://192.168.111.18:8000/hdp

redhat6 (HDP-UTILS-1.1.0.20):
http://192.168.111.18:8000/hdp

redhat7 (HDP-2.4):
http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.4.3.0

redhat7 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos7

suse11 (HDP-2.4):
http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/updates/2.4.0.0

suse11 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/suse11sp3

ubuntu12 (HDP-2.4):
http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x/updates/2.4.0.0

ubuntu12 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu12

ubuntu14 (HDP-2.4):
http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.4.0.0

ubuntu14 (HDP-UTILS-1.1.0.20):
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu12

Services:

HDFS
DataNode : 3 hosts
NameNode : hddb01
NFSGateway : 0 host
SNameNode : hddb02
YARN + MapReduce2
App Timeline Server : hddb02
NodeManager : 3 hosts
ResourceManager : hddb02
Tez
Clients : 2 hosts
Hive
Metastore : hddb02
HiveServer2 : hddb02
WebHCat Server : hddb02
Database : MySQL (New MySQL Database)
HBase
Master : hddb01
RegionServer : 3 hosts
Phoenix Query Server : 0 host
Pig
Clients : 2 hosts
Sqoop
Clients : 2 hosts
Oozie
Server : hddb02
Database :
ZooKeeper
Server : 3 hosts
Storm
DRPC Server : hddb01
Nimbus : hddb01
UI Server : hddb01
Supervisor : 3 hosts
Flume
Flume : 3 hosts
Ambari Metrics
Metrics Collector : hddb01
Atlas
Metadata Server : hddb01
Kafka
Broker : hddb01
Knox
Gateway : hddb01
Mahout
Clients : 2 hosts
Slider
Clients : 2 hosts
SmartSense
HST Server : hddb01
Spark
History Server : hddb01
Thrift Server : 0 host

 

 

分类上一篇:     分类下一篇:

Leave a Reply