What are the small scripts commonly used in Hadoop deployment

This article introduces the knowledge of "what are the small scripts commonly used in Hadoop deployment". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Recently abandoned the non-ssh connection of Hadoop cluster deployment, or returned to the way of using ssh key authentication. There is some trouble here. Every machine has to upload the public key. I am also a very lazy person, so write a few small scripts to complete, as long as on a machine can do the distribution of public keys.

The first step is to generate the ssh key script:

#! / bin/sh ssh-keygen-t rsa-P''- f ~ / .ssh/id_rsa cp ~ / .ssh/id_rsa.pub ~ / .ssh/authorized_keys

Generally speaking, ssh-keygen needs to enter passphrase, but there are usually three returns. I am lazy to enter and add-P''.

Then there is the script that adds the public key to the slave node:

#! / bin/sh read-p "enter remote server IP:" ip ssh-copy-id-o StrictHostKeyChecking=no-I ~ / .ssh/id_rsa.pub root@$ip ssh root@$ip 'sed-I "s / ^ # RSAAuthentication\ yes/RSAAuthentication\ yes/g" / etc/ssh/sshd_config' ssh root@$ip' sed-I "s / ^ # PubkeyAuthentication\ yes/PubkeyAuthentication yes/g" / etc/ssh/sshd_config' ssh root@$ip 'sed-I "s / ^ # PermitRootLogin\ yes/PermitRootLogin\ yes/g "/ etc/ssh/sshd_config' ssh root@$ip 'service sshd restart' hostname= `ssh root@$ {ip}' hostname' `echo" add hostname and IP to the local / etc/hosts file "echo" $ip $hostname "> > / etc/hosts echo" remote host name is $hostname Please check / etc/hosts to make sure that the hostname and IP are added to the host list file "echo" host public key copy complete "

Then the third script reads the list of hosts and copies / etc/hosts to all hosts:

#! / bin/sh cat / etc/hosts | while read LINE do ip= `echo $LINE | awk'{print $1}'| grep-v ":" | grep-v "127.0.0.1" `echo "Copying / etc/hosts to ${ip}" scp-o StrictHostKeyChecking=no / etc/hosts root@$ {echo}: / etc/ done "what are the mini scripts commonly used in Hadoop deployment? thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!