Isuru's Blog

The ELK stack  is a popular stack for searching and analyzing data. Many people use it for analyzing logs. WSO2 also has a full-fledged Big Data Analytics Platform, which can analyze logs and do many more things. In this blog post, I'm explaining on how to monitor logs with  Elasticsearch , Logstash and Kibana . I will mainly explain logstash configurations. I will not show how to set up Elasticsearch and Kibana. Those are very easy to setup and there are not much configurations. You can just figure it out very easily! :) If you want to test an elasticsearch server, you can just extract the elasticsearch distribution and start an elasticsearch server. If you are using Kibana 3, you need to use a web server to host the Kibana application. With Kibana 4, you can use the standalone server provided in the distribution. Configuring Logstash Logstash  is a great tool for managing events and logs. See  Getting Started with Logstash  if you haven't used ...

There is a proposal to  build performance probes in WSO2 Platform . For that I started looking in to some performance monitoring libraries. Following libraries were mentioned in the WSO2 architecture thread. Metrics Parfait JAMon (Java Application Monitor) While looking in to these libraries, I found out about following also. Java Simon - Simple Monitoring API Perf4J Here is a quick comparison of each project. These comparison criteria are based on the requirements in above proposal. Metrics Parfait JAMon Java Simon Perf4J License Apache License 2.0 Apache License 2.0 JAMon License New BSD License Apache License 2.0 Source GitHub Google Code Sourceforge GitHub GitHub Latest Version 3.1.0 0.2.8 2.79 4.0.0 0.9.16 Last Published Sep 4, 2014 Jun 01, 2011 Aug 20, 2014 Oct 29, 2014 Oct 16, 2011 Java Version - Java 6 - Java 7 - Counts Yes Yes Yes Yes No Timings Yes Yes Yes Yes Yes JMX Support Yes Yes No Yes Yes Enable/Disable No No* Yes Yes Yes* * Not ...

These days, I got the chance to play with ELK  ( Elasticsearch , Logstash & Kibana ). These tools are a great way to analyze & visualize all logs. You can easily analyze all wso2carbon.log files from ELK. However we also needed to use ELK for monitoring WSO2 products and this post explains the essential steps to use logstash JMX input plugin to monitor WSO2 servers. Installing Logstash JMX input plugin Logstash has many inputs and the JMX input plugin is available under "contrib" We can use "plugin install contrib" command to install extra plugins. cd /opt/logstash/bin sudo ./plugin install contrib Note: If you use logstash 1.4.0 and encounter issues in loading the jmx4r, please refer Troubleshooting below. Logstash JMX input configuration When using the JMX input plugin, we can use a similar configuration as follows. We are keeping the logstash configs in "/etc/logstash/conf.d/logstash.conf" input { jmx{ path ...