DengQN·一个普通程序员;
CDH 5.7.6 在ubuntu14.04的安装
2018-09-30 17:00 65
#机器#安装#使用#添加#源#数据库#过程#略
  • 记录安装cdh在ubuntu14.04的过程,不保证在所有机器上能无问题。。。

  • 以下命令推荐在su下运行

系统安装

系统配置

静态ip

编辑 /etc/network/interfaces

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto p2p1
iface p2p1 inet static
address 172.16.1.68
netmask 255.255.255.0
gateway 172.16.1.1

hosts

设置/etc/hosts,不用ip,直接使用名字

172.16.1.68     master
172.16.1.69     data1
172.16.1.70     data2

hostname

每台机器都在在/etc/hostname 添加将来要使用的机器名。

机器名要和hosts文件的名字一样。

ssh 无密码互通

~/.ssh,用ssh-kengen 生成密钥(过程中提问直接回车)。

ssh-copy-id [hostname]把密钥发送给别的机器

如:ssh-copy-id localhost 给自己发送一个

关闭防火墙

sudo ufw disable

sudo reboot

时间同步

apt-get install ntp

更新源

这里说的是cloudera的源,其他的略。

cd /etc/apt/sources.list.d
sudo wget http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/cloudera.list
sudo apt-get update

获取apt key

curl -s https://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key| sudo apt-key add - 

安装cloudera Server

在Cloudera Mananger 机器上

sudo apt-get -y --force-yes install cloudera-manager-daemons cloudera-manager-server

mysql

只在master上配置mysql,apt-get install mysql-server

  • 关闭mysql服务
  • 添加配置文件,vim /etc/mysql/conf.d/mysql_cloudera_manager.cnf

内容:

[mysqld]
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
# symbolic-links = 0

key_buffer = 16M
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M

#log_bin should be on a disk with enough free space. Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your system
#and chown the specified folder to the mysql user.
log_bin=/var/lib/mysql/mysql_binary_log

# For MySQL version 5.1.8 or later. Comment out binlog_format for older versions.
binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

sql_mode=STRICT_ALL_TABLES
  • 重启服务,service mysql start

  • 登陆mysql并建立数据库

create database amon DEFAULT CHARACTER SET utf8;  
grant all on amon.* TO 'amon'@'localhost' IDENTIFIED BY 'amon_password';  
grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'amon_password';  
create database smon DEFAULT CHARACTER SET utf8;  
grant all on smon.* TO 'smon'@'localhost' IDENTIFIED BY 'smon_password';  
grant all on smon.* TO 'smon'@'%' IDENTIFIED BY 'smon_password';  
create database rman DEFAULT CHARACTER SET utf8;  
grant all on rman.* TO 'rman'@'localhost' IDENTIFIED BY 'rman_password';  
grant all on rman.* TO 'rman'@'%' IDENTIFIED BY 'rman_password';  
create database hmon DEFAULT CHARACTER SET utf8;  
grant all on hmon.* TO 'hmon'@'localhost' IDENTIFIED BY 'hmon_password';  
grant all on hmon.* TO 'hmon'@'%' IDENTIFIED BY 'hmon_password';  
create database hive DEFAULT CHARACTER SET utf8;  
grant all on hive.* TO 'hive'@'localhost' IDENTIFIED BY 'hive_password';  
grant all on hive.* TO 'hive'@'%' IDENTIFIED BY 'hive_password';  
create database oozie;  
grant all privileges on oozie.* to 'oozie'@'localhost' identified by 'oozie';  
grant all privileges on oozie.* to 'oozie'@'%' identified by 'oozie'; 
  • 开启远程访问数据库,把my.cnfbind-host:localhost这一行注释了

JDK 安装

apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y install oracle-j2sdk1.7  

可能要--force-yes

master和agent都是要装,前提是添加了源。

cloudera-manager 初始化数据库

/usr/share/cmf/schema/scm_prepare_database.sh mysql -uroot -p --scm-host localhost scm scm scm_password 

master 开启cm

service cloudera-scm-server start

使用网页部署cdh

地址是 http://host:7180

登陆账户密码都是admin