Get the App
SLTechnology News&Howtos  ›  Internet Technology  › 

There is only one problem with hive distict and count operating reducer

Shulou Source: shulou.com Published: 2022-06-03 06:46:36 11月14日 Update

Select count (distinct *) from table

Writing like this will generate a job with only one reducer execution, which can be changed to the following:

Select count (*) from (

Select distinct * from table) t

In this way, two job can be generated, and multiple reducer can be executed, thus improving efficiency.

Tags: Generation only two multiple efficiency problems. Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno Redmi Shulou Technology macOS MySQL Linux