APIGEE: CI/CD Pipeline for API Proxies

Apigee

                                       In this article we will see how to create CI/CD pipeline for APIGEE API proxies. I have referred couple of articles on APIGEE community on the same topic. Those gave some idea on how to setup CI/CD pipeline for API proxies. Here are the tools which I have used to setup CI/CD.

  1. Jenkins
  2. NodeJs
  3. apigeelint
  4. newman
  5. APIGEE Management APIs

Make sure that you have created APIGEE edge account and a sample proxy to start with.  Below is the architecture diagram which shows CI/CD pipeline and the stages involved. You can use this as a baseline CI/CD for your projects and can enhance it based on your requirements.

APIGEE_CI_CD

Here are the steps I have implemented in CI/CD pipeline.

  1. Developer pushes the API proxy code to GIT.
  2. Jenkins polls GIT and starts CI/CD Stage 1 based on GIT changes.
  3. As part of Stage 1, the code will be pulled into workspace.
  4. In “Static Code Analysis” stage, the code will be analyzed for any violations of best code practices and anti-patterns usage. If this stage is success it proceeds with build stage. After the each stage completion either success or failure the notification will be sent to Slack channel.
  5. As part of build stage we will create APIGEE API proxy bundle.
  6. In Deploy stage I used APIGEE management APIS to deploy API proxy bundle.
  7. Once the deployment is successful, then the integration tests will be triggered. I used Newman to do integration testes. Newman requires integration tests collection file as input. The test cases can be created easily with Postman
  8. In all the stages the notifications will be triggered to Slack channel.

There are some enhancements which I will do in coming days. Below are some of the changes which I will target as enhancements.

  • Adding email, Hipchat notifications
  • Revert the API proxy to previous revision if the integration tests fail.
  • If integration tests success, promote build to load test environment and run load test scripts.

The setup and project used as part of this article is available on github. Till then, Spread love for APIs!!!

Apigee Message Logging Policy Demo

 

Advertisements
Tagged with:
Posted in APIGEE, Uncategorized

APIGEE API gateway log management with ELK(Elastic Search, Logstash and Kibana)

Apigee

                  In this article we will see how we can  manage logs generated by APIGEE API gateway with Elastic Search, Logstash and Kibana. The below diagram highlights how log data flows through ELK stack for data visualization and monitoring.

apigee_log_management

As part of API gateway we will use MessageLogging policy to capture API proxy logs at proxy flow, target flow and post client flow based on the requirement. As of today, MessageLogging policy  supports syslog and file based logging(available for only on-premise installations). Below is MessageLogging policy configuration to send logs to syslog server.

Now we will see logstash configuration to ingest data from syslog to logstash. Here I am not giving steps to setup ELK as that is out of scope.

Once syslog data is ingested successfully to Logstash we can configure visualization based on available fields in index. Below is sample visualization created for API proxy response status codes.

kibana_visualization

In coming articles we will discuss another topic. Till then, Spread love for APIs!!!

Tagged with: ,
Posted in APIGEE

APIGEE – An Introduction to API Gateway

Apigee

In this article I want to give brief introduction about APIGEE. Mainly APIGEE offers below functionalities out of the box as api gateway.

  1. Protocol Transformation
    Transform from or to any protocol including SOAP, REST, XML binary, or custom
  2. Traffic Management
    Flexible, distributed quota management, rate limiting, and spike arrest policies out-of-the-box
  3. API Security
    Built-in support for address filtering, JSON and XML schema validation, and bot detection
  4. Data Access & Security
    Two-way SSL/TLS, API key validation, OAuth1, OAuth2, SAML, CORS, encrypted store, and HIPAA and PCI compliance
  5. API Products
    Create different tiers by packaging APIs with varying rate limits and pricing
  6. API Analytics
    Fine-grained performance monitoring, including anomaly tracing and drill-down and usage metrics related to apps, developers, and APIs
  7. API Monetization
    Flexible rate plans, international billing, and usage tracking
  8. Global Policy Management
    Enforce consistent security and governance policies across all APIs
  9. Developer Portal
    A customizable portal for API providers to manage developers, APIs, and API documentation and versioning

APIGEE sits between service consumers and backend services. As an API gateway, APIGEE takes care of common functionalities required for the APIs. Hence backend services can concentrate only on the core business logic. The below digram  depicts where exactly APIGEE fits.

APIGEE

Now we will see how the request and response goes through APIGEE. Every client request will go through proxy and target endpoints where we will attach APIGEE policies. Policy is nothing but a simple module which will provide common API functionality which we can configure using XML. In each flow APIGEE exposes lot of flow variables.

APIGEE (2).png

Below are list of policies available out of the box which you can attach to proxy and target flows.

APIGEE (3)

If you want to experience APIGEE edge, follow below steps to create trail account.

  1. Goto https://login.apigee.com/sign_upEnter the required information to create account with APIGEE.
  2. After activating APIGEE account, login to https://login.apigee.com/login access Apigee Edge management UI

Below are some of the reference materials to deep dive into APIGEE

 

Tagged with:
Posted in APIGEE

APIGEE – Can we configure parameters for Message Logging Policy?

Apigee

Apigee MessageLogging policy has limitation to use configurable parameters such as syslog server host, port etc… details. If we are not going to configure these parameters we may get into trouble while moving the proxy from one environment to other environment. To achieve the portability the approach is to have MessageLogging policy for each environment. Based on the environment in which the proxy is running the policy will be applied. Below is the sample proxy with message logging policy for each environment. The proxy definition is given below.

The test and prod environment MessageLogging policy configuration is given below.

The proxy demonstrated is available on GitHub to download and play with it.

Apigee Message Logging Policy Demo

Tagged with:
Posted in APIGEE

APIGEE – How To Handle Base64 Encoding Decoding?

Apigee

In this article we will see how to encode and decode base64 strings while building APIGEE proxies. As part of APIGEE we have BasicAuthentication policy which deals with base64 encoded authorization header. But if we want to deal with any base64 encoded string other than Authorization header  we should go with JavaScript policy or JavaCallout policy or PythonScript policy custom implementation. In this article I will show you how to achieve base64 encode and decode using JavaScript policy.

Let me create a simple proxy with JavaScript policy to decode base64 encoded string. Below is the JavaScript policy configuration.

In the JavaScript policy I have included the Base64EncodeDecode js file which perform the encode and decode. Below is the JavaScript to decode base64 encoded string.

The JavaScript which does base 64 encode and decode is available here.

The sample proxy created to demonstrate base64 encode and decode is available on GitHub. Download the sample proxy bundle and import to APIGEE Edge to play with it.

APIGEE

In next article we will discuss another use case. Till then Happy Coding!!!

 

Tagged with: ,
Posted in APIGEE

Java 11 Features – Java Flight Recorder

Java 11

                            In this article we will see how we can leverage Java Flight Recorder feature as part of Java 11. Earlier it was one of the commercial feature. But with Java 11 with JEP 328 this is open sourced to OpenJDK from OracleJDK. The Java Flight Recorder records the OS and JVM events to a file which can be inspected using Java Mission Control (JMC). Enabling JFR puts minimal overhead on the JVM performance. Hence this can be enabled for production deployments too. Now we will see some of the JVM arguments to enable JFR.

  • Time Based

  • Continuous with dump on demand

  • Continuous with dump on exit

As the JFR is built in available with Java 11, this excites the developer community. We can reduce the dependency on 3rd party profilers as well.

As part of Java 11 we are getting jdk.jfr module. This API allows programmers to produce custom JFR events and consume the JFR events stored in a file to troubleshoot the issue.

You can download the Java11 early access from http://jdk.java.net/11/ to explore the features.

Tagged with: ,
Posted in Java

Java 10 – Local Variable Type Inference

Java10
In this article we will see Java10 feature called Local Variable Type Inference proposed as part of JEP 286. From the first version of Java it is strongly typed language where we need to mention each variable data type. We all were feeling Java is verbose language and expecting precise, compact way of writing Java code. Java 8 addressed this concern some what. Java 10 added Local Variable Type Inference with initializer to eliminate verbosity. For example,

jshell> Map<String,String> map = new HashMap<>();
jshell> var map = new HashMap<>(); //This is valid with Java10

Here LHS variable datatype will be determined by RHS statement. For example,

jshell> var i = 3;
i ==> 3 //based on RHS, the LHS datatype is int.
jshell>int i=3,j=4; //Valid Declaration
but,
jshell> var j=4,k=5; //Not a Valid Declaration
| Error:
|'var' is not allowed in a compound declaration
| var j=4,k=5;
|^

You can use this feature for enhanced for loop and for loop as well.

jshell> List names = Arrays.asList("ABC","123","XYZ");
names ==> [ABC, 123, XYZ]
jshell> for(var name : names){
...> System.out.println("Name = "+ name);
...> }

Name = ABC
Name = 123
Name = XYZ

We can use Local Variable Type Inference in the for loop as well.


jshell> int[] arr = {1,2,3,4};
arr ==> int[4] { 1, 2, 3, 4 }

jshell> for (var i=0;i<arr.length;i++){
   ...> System.out.println("Value = "+i);
   ...> }
Value = 0
Value = 1
Value = 2
Value = 3

There are certain scenarios where this feature is not valid to use. For example,

  • Not valid for constructor variables
  • Not valid for instance variables
  • Not valid for method parameters
  • Not valid to assign NULL value
  • Not valid as return type

Let us see examples for above statements.


jshell> public class Sample {
   ...>    private var name = "xyz";
   ...>    public Sample(var name) {
   ...>     this.name=name;
   ...>    }
   ...>    public void printName(var name){
   ...>      System.out.println(name);
   ...>    }
   ...>    public var add(int a, int b) {
   ...>     return a+b;
   ...>    }
   ...> }
|  Error:
|  'var' is not allowed here
|     private var name = "xyz"; //Instance variable
|             ^-^
|  Error:
|  'var' is not allowed here
|     public Sample(var name) { //Constructor variable
|                   ^-^
|  Error:
|  'var' is not allowed here
|     public void printName(var name){ //Method parameter
|                           ^-^
|  Error:
|  'var' is not allowed here
|     public var add(int a, int b) { //Method return type
|            ^-^


jshell> public class Sample {
   ...>    
   ...>    public static void main(String[] args) {
   ...>     var s = null;
   ...>    }
   ...> }
|  Error:
|  cannot infer type for local variable s
|    (variable initializer is 'null')
|      var s = null;
|      ^-----------^

When we migrate the code from lower versions to Java10, we no need to worry about the Local Variable Type Inference as this has the backward compatibility.

In the coming post we will learn another topic. Till then stay tuned!

Posted in Java
DZone

DZone MVB

Java Code Geeks
Java Code Geeks