Monitor Apache Kafka using AppManager

0 Likes
over 4 years ago
To monitor Apache Kafka using AppManager you need to use Kafka-Monitor and Java Application Server on Windows and UNIX module. Kafka-Monitor is a framework that allows you to monitor Kafka cluster using end-to-end pipelines to obtain a number of stats such as service availability.

Kafka-Monitor and steps to configure it are available at GitHub at the following location:
https://github.com/linkedin/kafka-monitor

Once the kafka-monitor has been deployed and configured we need to use AppManager for Java module to discover the Kafka.

Steps to discover Kafka




  1. Download and install Java Application Server on Windows and UNIX 8.0 module. It is available at the following location:
    https://www.netiq.com/Support/am/trialcenter.asp




  1. Run the discovery KS for Java and in the field “Jolokia URL address” input the address for Jolokia as:

    {IP address of Kafka server}:8778




2


  1. Once the job stops, Kafka will be discovered in JAVA view and you can see the tree structure with managed beans.



3


  1. Now we can run the “ManagedBeanUtil” KS to monitor various parameters of Kafka.

  • E.g. To monitor the kafka.consumer/consumer-fetch-manager-metrics/kmf-consumer/fetch-rate we deploy the KS on its respective component and specify the max-min thresholds, as shown below:



5


  1. We get an event when the thresholds are crossed, as below:


6


  1. Some of the important parameters to monitor can be found here:
    https://blog.serverdensity.com/how-to-monitor-kafka/




 

Labels:

How To-Best Practice
Comment List
Anonymous
Related Discussions
Recommended