Get the App
SLTechnology News&Howtos  ›  Internet Technology  › 

HDFS benchmark Test in HADOOP

Shulou Source: shulou.com Published: 2022-06-03 07:07:41 11月23日 Update

hdfs benchmark test: disk read and write performance test hadoop jar ./ hadoop-mapreduce-client-jobclient-2.7.2.3.jar TestDFSIO -read -nrFiles 10 -fileSize 10MBhadoop jar ./ hadoop-mapreduce-client-jobclient-2.7.2.3.jar TestDFSIO -write -nrFiles 10 -fileSize 10MBdev Environment:

read:19/06/28 08:14:45 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read19/06/28 08:14:45 INFO fs.TestDFSIO: Date & time: Fri Jun 28 08:14:45 CST 201919/06/28 08:14:45 INFO fs.TestDFSIO: Number of files: 1019/06/28 08:14:45 INFO fs.TestDFSIO: Total MBytes processed: 100.019/06/28 08:14:45 INFO fs.TestDFSIO: Throughput mb/sec: 1369.8630136986319/06/28 08:14:45 INFO fs.TestDFSIO: Average IO rate mb/sec: 1448.01586914062519/06/28 08:14:45 INFO fs.TestDFSIO: IO rate std deviation: 313.016840947768119/06/28 08:14:45 INFO fs.TestDFSIO: Test exec time sec: 1.444

write:

19/06/28 08:17:58 INFO fs.TestDFSIO: ----- TestDFSIO ----- : write19/06/28 08:17:58 INFO fs.TestDFSIO: Date & time: Fri Jun 28 08:17:58 CST 201919/06/28 08:17:58 INFO fs.TestDFSIO: Number of files: 1019/06/28 08:17:58 INFO fs.TestDFSIO: Total MBytes processed: 100.019/06/28 08:17:58 INFO fs.TestDFSIO: Throughput mb/sec: 94.250706880301619/06/28 08:17:58 INFO fs.TestDFSIO: Average IO rate mb/sec: 94.3214874267578119/06/28 08:17:58 INFO fs.TestDFSIO: IO rate std deviation: 2.578073452981487719/06/28 08:17:58 INFO fs.TestDFSIO: Test exec time sec: 2.48219/06/28 08:17:58 INFO fs.TestDFSIO:

Tags: Testing benchmarking performance environment disk Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno NVidia MariaDB Microsoft Shulou Information OPPO Reno