APIGEE API gateway log management with ELK(Elastic Search, Logstash and Kibana)

Apigee

                  In this article, we will see how we can manage logs generated by APIGEE API gateway with Elastic Search, Logstash and Kibana. The below diagram highlights how log data flows through ELK stack for data visualization and monitoring.

apigee_log_management

As part of the API gateway, we will use MessageLogging policy to capture API proxy logs at proxy flow, target flow and post client flow based on the requirement. As of today, MessageLogging policy supports Syslog and file-based logging(available for only on-premise installations). Below is MessageLogging policy configuration to send logs to the syslog server.

Now we will see logstash configuration to ingest data from syslog to logstash. Here I am not giving steps to setup ELK as that is out of scope.

Once Syslog data is ingested successfully to Logstash we can configure visualization based on available fields in the index. Below is sample visualization created for API proxy response status codes.

kibana_visualization

In coming articles we will discuss another topic. Till then, Spread love for APIs!!!

Advertisements

I am Siva Prasad Rao Janapati. Working as Technical Architect. Has hands on experience on ATG Commerce(DAS/DPS/DCS), Mozu commerce, Broadleaf Commerce, Java, JEE, Spring, Play, JPA, Hibernate, Velocity, JMS, Jboss, Weblogic,Tomcat, Jetty, Apache, Apache Solr, Spring Batch, JQuery, NodeJS, SOAP, REST, MySQL, Oracle, Mongo DB, Memcached, HazelCast, Git, SVN, CVS, Ant, Maven, Gradle, Amazon Web services, Rackspace, Quartz, JMeter, Junit, Open NLP, Facebook Graph,Twitter4J, YouTube Gdata, Bazzarvoice,Yotpo, 4-Tell, Alatest, Shopzilla, Linkshare. I have hands on experience on open sources and commercial technologies.

Tagged with: ,
Posted in APIGEE

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

DZone

DZone MVB

Java Code Geeks
Java Code Geeks
%d bloggers like this: