How to install hue and integrate hive hdfs oozie
How to carry out hue installation and hive hdfs oozie integration, I believe that many inexperienced people do not know what to do, so this article summarizes the causes of the problem and solutions, through this article I hope you can solve this problem.
[toc]
Environmental requirements jdk
Maven
Git
Third party relies on yum install-y gcc-c++ libxml2-devel.x86_64 libxslt-devel.x86_64 python-devel openldap-devel asciidoc cyrus-sasl-gssapi openssl-devel mysql-devel sqlite-devel gmp-devel libffi-devel npm. What else is missing to install on your own?
Create hue users and user groups
Groupadd hue
Useradd-g hue hue
Switch users and enter the hue root directory
Su hue
Cd ~
Pull hue source code from git
Git clone https://github.com/cloudera/hue.git
Go to the hue-master directory and compile
Cd hue-master
Make apps needs to wait for a long time. If there is a lack of dependencies in the middle, you can install it yourself.
Modify hue configuration file
Vi / home/hue/hue-master/desktop/conf/pseudo-distributed.ini
Time zone
[desktop]
Time_zone=Asia/Shanghai
User role
[desktop]
Server_user=hue
Server_group=hue
Default_user=hue
Hdfs configuration
[hadoop]
[[hdfs_clusters]]
Fs_defaultfs=hdfs:// cluster name
Webhdfs_url= http://hdfsMaster1:14000/webhdfs/v1
Yarn configuration
[hadoop]
[[yarn_clusters]]
[default]
Resourcemanager_host=resourcemanager
Resourcemanager_api_url= http://resourcemanager:8088
Submit_to=True
Proxy_api_url= http://172.19.131.104:8088
History_server_api_url= http://172.19.131.104:19888
Database configuration
(hue database is created first and permissions are granted to hue users) [database]
Engine=mysql
Host=mysqlhost
Port=3306
User=hue
Password=123456
Name=hue
Oozie configuration (default but not matched)
[oozie]
Remote_data_dir=/user/hue/oozie/workspaces
Enable_cron_scheduling=true
Enable_document_action=true
Liboozie configuration
[liboozie]
Oozie_url= http://master1:11000/oozie
Beeswax configuration (hive)
[beeswax]
Hive_server_host=172.19.131.107
Hive_server_port=10000
Database initialization
/ home/hue/hue-master/build/env/bin/hue syncdb
/ home/hue/hue-master/build/env/bin/hue migrate
Running
/ home/hue/hue-master/build/env/bin/supervisor
Problem solving if the hdfs path is inaccessible
Configure hadoop.proxyuser.hue.groups=* under the core-site configuration file of hadoop
Hadoop.proxyuser.hue.hosts=*
If the hive database cannot be accessed
You can authorize the database of hive at the rights management place (I use ranger).
If oozie submits the task and fails to execute it
You can view the log of MapReduce. It may be that the current user cannot submit task configuration information to the directory under yarn.
After reading the above, have you mastered how to install hue and how to integrate hive hdfs oozie? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!
