Shulou(Shulou.com)06/01 Report--

This article mainly introduces how Apache Hudi integrates Prometheus/Pushgateway for monitoring. The article is very detailed and has certain reference value. Interested friends must read it!

1. availability

In Hudi's newest master branch, prometheus monitors Hudi application metrics, which will be available in the upcoming version 0.6.0.

2. profile

Prometheus is a popular surveillance service. In the upcoming Apache Hudi 0.6.0 release, in addition to the existing reporter types (Graphite, JMX, DATADOG, user-defined metrics), we will introduce the ability to report Hudi metrics via pushgateway and httpserver.

3. Configuration 3.1 pushgateway

Similar to other supported reporters, enabling pushgateway reporters requires configuring the following properties.

hoodie.metrics.on=true

hoodie.metrics.reporter.type=PROMETHEUS_PUSHGATEWAY

The following properties are used to configure the address and port number of the pushgateway. Address default is localhost, port default is 9091

hoodie.metrics.pushgateway.host=x.x.x.x

hoodie.metrics.pushgateway.port=9091

The following properties are used to configure whether to delete monitoring information from pushgateway at the end of the task. The default is true.

hoodie.metrics.pushgateway.delete.on.shutdown=false

The following attributes are used to configure the task name prefix and whether a random suffix is required. The default is true.

hoodie.metrics.pushgateway.job.name=xiaomm

hoodie.metrics.pushgateway.random.job.name.suffix=false

The following properties are used to configure whether to delete monitoring information from pushgateway at the end of the task. The default is true.

hoodie.metrics.pushgateway.delete.on.shutdown=false3.2 prometheus-httpserverhoodie.metrics.on=true

hoodie.metrics.reporter.type=PROMETHEUS

hoodie.metrics.reporter.host=9090

This concludes the configuration description

4. example demonstrates

In this example, we run a HoodieDeltaStreamer with metric collection enabled and configured accordingly.

The above is "Apache Hudi how to integrate Prometheus/Pushgateway monitoring" all the content of this article, thank you for reading! Hope to share the content to help everyone, more relevant knowledge, welcome to pay attention to the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.