Install CDH with cm5.16 in centos7.4 environment
Modify ip and restart / etc/init.d/netwrok restart
PROXY_METHOD=noneBROWSER_ONLY=noBOOTPROTO=staticDEFROUTE=yesIPV4_FAILURE_FATAL=noIPV6INIT=noIPV6_AUTOCONF=noIPV6_DEFROUTE=noIPV6_FAILURE_FATAL=noIPV6_ADDR_GEN_MODE=stable-privacyNAME=ens32UUID=a0ad438d-a586-4c30-b624-601b342a39c4DEVICE=ens32ONBOOT=yesNETMASK=255.255.255.0IPADDR=192.168.52.130GATEWAY=192.168.52.2DNS1=223.5.5.5
Modify hostname
Hostnamectl set-hostname bigdata-cdh01.kfk.com
Modify native DNS / etc/host
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4#::1 localhost localhost.localdomain localhost6 localhost6.localdomain6192.168.52.130 bigdata-cdh01.kfk.com bigdata-cdh01192.168.52.131 bigdata-cdh02.kfk.com bigdata-cdj02192.168.52.132 bigdata-cdh03.kfk.com bigdata-cdh03
Turn off the firewall
Systemctl stop firewalld.servicesystemctl disable firewalld.service (boot does not start) systemctl status firewalld.service
Disable selinux cat / etc/sysconfig/selinux
SELINUX=disabled
Temporarily shut down selinux
Getenforcesetenforce 0getenforce
Disable ipv6
Echo "> > / etc/modprobe.d/dist.confecho" alias net-pf-10 off "> > / etc/modprobe.d/dist.confecho" alias ipv6 off "> > / etc/modprobe.d/dist.conf
Uninstall JDK
Rpm-qa | grep javarpm-e-- nodeps xxx
Set the number of file openings and the maximum number of processes for users
Number of file openings
Ulimit-a
Maximum number of processes for users
Ulimit-u
Setup / etc/security/limits.conf
* soft nofile 32768 * hard nofile 1048576 * soft nproc 65536 * hard nporc unlimited* soft memlock unlimited* hard memlock unlimited
Ssh mutual trust
Ssh-keygen-t rsatouch authorized_keyschmod 600authorized_keyscat id_rsa.pub > authorized_keysssh-copy-id-I id_rsa.pub root@192.168.52.130
Using ntp for time synchronization
Primary node
Driftfile / var/lib/ntp/driftrestrict 192.168.52.130 nomodify notrap nopeer noquery restrict 127.0.0.1restrict:: 1restrict 192.168.52.2 mask 255.255.255.0 nomodify notrapserver 127.127.1.0Fudge 127.127.1.0 stratum 10includefile / etc/ntp/crypto/pwkeys / etc/ntp/keysdisable monitor
Slave nodes, other nodes are the same
Driftfile / var/lib/ntp/driftrestrict 192.168.52.131 nomodify notrap nopeer noquery restrict 127.0.0.1restrict:: 1restrict 192.168.52.2 mask 255.255.255.0 nomodify notrapserver 192.168.52.130Fudge 192.168.52.130 stratum 10includefile / etc/ntp/crypto/pwkeys / etc/ntp/keysdisable monitor
Shut down ntpd first
Ntpdate 192.168.52.130service ntpd restartsystemctl enable ntpd
Ntpq-p checks whether it is synchronized. A * indicates that it has been synchronized.
Turn off transparent large pages and swap memory
Echo never > / sys/kernel/mm/transparent_hugepage/defrag echo never > / sys/kernel/mm/transparent_hugepage/enabled echo "echo never > / sys/kernel/mm/transparent_hugepage/enabled" > > / etc/rc.local echo "echo never > / sys/kernel/mm/transparent_hugepage/defrag" > > / etc/rc.local chmod + x / etc/rc.d/rc.local echo "vm.swappiness=1" > > / etc/sysctl.conf sysctl-p / / to make the configuration effective
Build a yum source
Rpm-ivh httpd-2.4.6-80.el7.centos.x86_64.rpm httpd-tools-2.4.6-80.el7.centos.x86_64.rpmmount / dev/sr0 / mntln-s / mnt centos7/etc/ yum.repos.d [CentOS-7] name=myrepobaseurl= http://bigdata-cdh01/centos7/enabled=1gpgcheck=0yum clean allyum makecacheyum install-y createrepocd / var/www/html/cm-5.16createrepo .cat myrepo.repo [myrepo] name=myrepobaseurl= http://bigdata-cdh01/cm-5.16/enabled=1gpgcheck=0
Install the download package for mysql5.7 database official website (mysql-5.7.13-linux-glibc2.5-x86_64.tar.gz)
Groupadd mysqluseradd mysql- s / sbin/nologin-M-g mysqltar-zxvf mysql-5.7.13-linux-glibc2.5-x86_64.tar.gz mv mysql-5.7.13-linux-glibc2.5-x86_64 / usr/local/mysqlchown mysql:mysql-R mysql/mkdir datachown-R mysql:mysql data/yum install-y perl-Data-Dumper.x86_64yum install libaio*./bin/mysqld-- initialize-- user=mysql-- basedir=/usr/local/mysql-- datadir=/usr/local / mysql/data/scp-r support-files/mysql.server / etc/init.d/mysqldvim / etc/init.d/mysqld basedir=/usr/local/mysqldatadir=/usr/local/mysql/dat Editing / etc/ my.cnf [mysqld] user=mysqldatadir=/usr/local/mysql/datasocket=/tmp/mysql.socklog-error=/usr/local/mysql/data/error.logpid-file=/usr/local/mysql/data/mysql/mysql.pidtmpdir=/tmp Settings mysql boot chkconfig mysqld on change password alter user user () identified by "123456" Flush privileges; opens remote login GRANT ALL PRIVILEGES ON *. * TO 'root'@'%' IDENTIFIED BY' 123456' WITH GRANT OPTION;flush privileges; to view the user name and password of mysql select host,user,authentication_string from mysql.user
Create a database
Create database metastore default character set utf8;CREATE USER 'hive'@'%' IDENTIFIED BY' 123456 has made Grant ALL PRIVILEGES ON metastore. * TO 'hive'@'%';FLUSH PRIVILEGES;create database hive default character set utf8;CREATE USER' hive'@'%' IDENTIFIED BY '123456 Taiwan Grant ALL PRIVILEGES ON hive. * TO 'hive'@'%';FLUSH PRIVILEGES;create database cm default character set utf8;CREATE USER' cm'@'%' IDENTIFIED BY '123456 Taiwan Grant ALL PRIVILEGES ON cm. * TO 'cm'@'%';FLUSH PRIVILEGES;create database am default character set utf8;CREATE USER' am'@'%' IDENTIFIED BY '123456 Taiwan Grant ALL PRIVILEGES ON am. * TO 'am'@'%';FLUSH PRIVILEGES;create database rm default character set utf8;CREATE USER' rm'@'%' IDENTIFIED BY '123456 Taiwan Grant ALL PRIVILEGES ON rm. * TO 'rm'@'%';FLUSH PRIVILEGES;create database hue default character set utf8;CREATE USER' hue'@'%' IDENTIFIED BY '123456 Taiwan Grant ALL PRIVILEGES ON hue. * TO 'hue'@'%';FLUSH PRIVILEGES;create database oozie default character set utf8;CREATE USER' oozie'@'%' IDENTIFIED BY '123456 Taiwan Grant ALL PRIVILEGES ON oozie. * TO 'oozie'@'%';FLUSH PRIVILEGES
Copy the package of jdbc, all nodes need to be installed
Cd / usr/share/java/ln-s mysql-connector-java-5.1.38-bin.jar mysql-connector-java.jar
Install Metabase
/ usr/share/cmf/schema/scm_prepare_database.sh mysql cm cm 123456
Install jdk from the command line
Yum list | grep oracleyum install-y oracle-j2sdk1.7.x86_64export JAVA_HOME=/usr/java/jdk1.7.0_67-clouderaexport JRE_HOME=$JAVA_HOME/jreexport CLASSPATH=$JAVA_HOME/libexport PATH=$PATH:$JAVA_HOME/bin
Install on the primary node
Yum install-y cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent
Install from nod
Yum install-y cloudera-manager-daemons cloudera-manager-agent
Upload CDH files
CDH-5.16.2-1.cdh6.16.2.p0.8-el7.parcel CDH-5.16.2-1.cdh6.16.2.p0.8-el7.parcel.sha1 manifest.jsonmv CDH-5.16.2-1.cdh6.16.2.p0.8-el7.parcel.sha1 CDH-5.16.2-1.cdh6.16.2.p0.8-el7.parcel.sha
Start
Systemctl start cloudera-scm-server
Do not choose
Add Servic
Kafka installation
Upload http://archive.cloudera.com/kafka/parcels/3.1.1/KAFKA-3.1.1-1.3.1.1.p0.2-el7.parcelKAFKA-3.1.1-1.3.1.1.p0.2-el7.parcel.shamanifest.json to / opt/cloudera/parcel-repo upload KAFKA-1.2.0.jar to / opt/cloudera/csd
