[hadoop] hive installation practice
1. Download the Hive installation package:
Download from the official website: http://hive.apache.org/downloads.html
two。 Upload the tar package of Hive and decompress it:
It is recommended to use the hadoop directory at the first level, which is convenient for subsequent use.
Decompress: tar-zxvf apache-hive-1.2.1-bin.tar.gz-C / home/hadoop/hive
Modify the name of the decompressed file: mv apache-hive-1.2.1-bin hive-1.2.1
3. Install MySql:
MySQL is used to store Hive metadata (see the previous article for installation tutorials)
4. Modify the configuration file: mainly configure metastore (metadata Storage) storage mode
4.1. Vi / home/hadoop/hive/hive-1.2.1/conf/hive-site.xml (storage method: embedded Derby, local mysql, remote mysql)
4.2 paste the following:
Javax.jdo.option.ConnectionURL jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true JDBC connect string for a JDBC metastore javax.jdo.option.ConnectionDriverName com.mysql.jdbc.Driver Driver class name for a JDBC metastore javax.jdo.option.ConnectionUserName root username to use against metastore database javax.jdo.option.ConnectionPassword root password to use against metastore database
5. Copy the jar package:
Copy the jar package of mysql driver to the lib directory of Hive
Download path: https://pan.baidu.com/s/17iHOIjt4XZbRAngGFf_GgA
6. Start Hive:
(1) you need to start the Hadoop cluster before starting Hive.
(2) use hadoop users
Start the command: / usr/local/src/hive-1.2.1/bin/hive
The following indicates that the startup is successful:
Hive >
7. Verify that Hive is running properly: enter the following command after starting Hive:
Hive > show databases
OK
Default
Test_db
Time taken: 0.567 seconds, Fetched: 2 row (s)
Hive > use default
OK
Time taken: 0.068 seconds
Hive > show tables
OK
Time taken: 0.086 seconds
8. Create a database, and the data files of the database are stored under / user/hive/warehouse/test_db.db in HDFS
Hive > create database test_db
OK
Time taken: 0.505 seconds
9. Create a table in test_db, and the data files of the table are stored under / user/hive/warehouse/test_db.db/t_test in HDFS
And the data file field of the table is separated by "|";
Use test_db
Create table flat1_test (mobile string,opr_type string,lastupdatetime string,monthly string,sp_code string,oper_code string,unknown string,subtime string)
Row format delimited
Fields terminated by'|'
10. Upload the data file to the hdfs specified directory, and the directory is the hive database table file directory.
Hadoop fs-put hivefile1.txt / user/hive/warehouse/test_db.db/flat1_test
11. Use sql to query data
Hive > select * from flat1_test
12. Query the metadata of Hive and query it in mysql.
Mysql > show databases;+-+ | Database | +-+ | information_schema | | hive | | mysql | | performance_schema | | test | +-+ 5 rows in set (0.00 sec) mysql > use hive Reading table information for completion of table and column namesYou can turn off this feature to get a quicker startup with-ADatabase changedmysql > mysql > show tables +-- + | Tables_in_hive | +-+ | BUCKETING_COLS | | CDS | | COLUMNS_V2 | | DATABASE_PARAMS | | DBS | | FUNCS | | FUNC_RU | | GLOBAL_PRIVS | | IDXS | | INDEX_PARAMS | | PARTITIONS | | PARTITION_KEYS | | PARTITION_KEY_VALS | | PARTITION_PARAMS | | PART_COL_PRIVS | | PART_COL_STATS | | PART_PRIVS | | | ROLES | | SDS | | SD_PARAMS | | SEQUENCE_TABLE | | SERDES | | SERDE_PARAMS | | SKEWED_COL_NAMES | | SKEWED_COL_VALUE_LOC_MAP | | SKEWED_STRING_LIST | | SKEWED_STRING_LIST_VALUES | | SKEWED_VALUES | | | SORT_COLS | | TABLE_PARAMS | | TAB_COL_STATS | | TBLS | | TBL_COL_PRIVS | | TBL_PRIVS | | VERSION | +-+ 35 rows in set (0.01 sec) mysql > select * from DBS | +- -+ | DB_ID | DESC | DB_LOCATION_URI | NAME | OWNER_NAME | OWNER_TYPE | +- -- + | 1 | Default Hive database | hdfs://XXXXXXXXXX:9000/user/hive/warehouse | default | public | ROLE | | 6 | NULL | | hdfs://XXXXXXXXXX:9000/user/hive/warehouse/test_db.db | test_db | hadoop | USER | +-+ -+ 2 rows in set (0.00 sec) mysql >
