Shulou(Shulou.com)05/31 Report--

This article shares with you the content of the sample analysis of python udf in hive. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

Udfdata.txt:

Aa pingpong,swim,running

Create table udfdata (name String,hobby String) row format delimited fields terminated by'\ t'; LOAD DATA LOCAL INPATH'/ Users/lifei/hivedata/udfdata.txt' OVERWRITE INTO TABLE udfdata;add file / Users/lifei/hivedata/parsehobby.py;select transform (name,hobby) using 'python parsehobby.py' as (name,hobby) from udfdataaa pingpongaa swimaa runningadd file / Users/lifei/hivedata/splithobby.py;select transform (hobby) using' python splithobby.py' as (name,hobby,test) from udfdata Pingpong swim running#parsehobby.pyimport sysfor line in sys.stdin: detail = line.strip (). Split ('\ t') if (len (detail) < 2): continue name = detail [0] hobbis = detail [1] hobbyArray = hobbis.split (',') for hobby in hobbyArray: print "% s\ t% s"% (name,hobby) # splithobby.py import sysfor line in sys.stdin: detail = line.strip (). Split (' ') print "% s\ t% s\ t% s"% (detail [0], detail [1], detail [2])

Ps: explode and collect_set

Chinese or str Times error: 'ascii' codec can't encode characters in position 61-62

Reload (sys)

Sys.setdefaultencoding ('utf8')

Thank you for reading! This is the end of this article on "sample Analysis of python udf in hive". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.