What most people are searching on google? Lets explore google hot trends…

                           spring batch

                         In this article we will see an second example Spring batch application. To create sample Spring Batch application, we will make use of google hot searches feed. Google provides hot trends app where you can choose the country to find the hot searches happening.

Google Hot

                 The google hot searches are available as atom feed(http://www.google.com/trends/hottrends/atom/feed?pn=p3). To use Spring Batch framework, we should add the below dependencies to pom.xml.

<dependency>
    <groupId>org.springframework.batch</groupId>
    <artifactId>spring-batch-core</artifactId>
    <version>3.0.3.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-beans</artifactId>
    <version>4.1.4.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
    <version>1.2.2.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-context</artifactId>
    <version>4.1.4.RELEASE</version>
</dependency>

We have used feed4j  to parse the atom feed. Download the feed4j jars and keep the dependencies in pom.xml.

<dependency>
    <groupId>feed4j</groupId>
    <artifactId>feed4j</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/feed4j.jar</systemPath>
</dependency>
<dependency>
    <groupId>nekohtml</groupId>
    <artifactId>nekohtml</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/nekohtml.jar</systemPath>
</dependency>
<dependency>
    <groupId>xercesImpl</groupId>
    <artifactId>xercesImpl</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/xercesImpl.jar</systemPath>
</dependency>
<dependency>
    <groupId>xml-apis</groupId>
    <artifactId>xml-apis</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/xml-apis.jar</systemPath>
</dependency>

Now, we are ready to implement Item Reader to get the hottrends feed and parse it.

package org.smarttechie.reader;

import it.sauronsoftware.feed4j.FeedParser;
import it.sauronsoftware.feed4j.bean.Feed;
import it.sauronsoftware.feed4j.bean.FeedItem;

import java.net.URL;
import java.util.ArrayList;
import java.util.List;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.smarttechie.model.GoogleTrendItem;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.NonTransientResourceException;
import org.springframework.batch.item.ParseException;
import org.springframework.batch.item.UnexpectedInputException;

/**
* The reader gets the atom feed and parses it.
* @author Siva Janapati
* @version 1.0
*
*/
public class GoogleTrendsReader implements ItemReader<List<GoogleTrendItem>>{

private Log log = LogFactory.getLog(GoogleTrendsReader.class);

/**
* The reader will make call to google hot trends API
*/
@Override
public List<GoogleTrendItem> read() throws Exception, UnexpectedInputException,
ParseException, NonTransientResourceException {
          URL url = new URL("http://www.google.com/trends/hottrends/atom/feed?pn=p3");
          Feed feed = FeedParser.parse(url);
          int items = feed.getItemCount();
          log.debug("The number of hot trends " + items);
          List<GoogleTrendItem> trends = null;
          if (items > 0) {
            trends = new ArrayList<GoogleTrendItem>();
              for (int i = 0; i < items; i++) {
                   FeedItem item = feed.getItem(i);
                   GoogleTrendItem trend = new GoogleTrendItem();
                   trend.setTitle(item.getTitle());
                   trend.setLink(item.getLink().toString());
                   trend.setDescription(item.getDescriptionAsText());
                   trends.add(trend);
        }
    }
    return trends;
  }
 }

The Item Writer implementation is given below.


package org.smarttechie.writer;

import java.util.List;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.smarttechie.model.GoogleTrendItem;
import org.springframework.batch.item.ItemWriter;

/**
* The writer to write the list of google trends
* @author Siva Janapati
* @version 1.0
*
*/
public class GoogleTrendsWriter implements ItemWriter<List<GoogleTrendItem>>{

private Log log = LogFactory.getLog(GoogleTrendsWriter.class);

@Override
public void write(List<? extends List<GoogleTrendItem>> trends)
throws Exception {
        if (trends != null && !trends.isEmpty()) {
           List<GoogleTrendItem> items = trends.get(0);
              if (items != null && !items.isEmpty()) {
                 for (GoogleTrendItem trend : items) {
                   log.debug("Title: " + trend.getTitle());
                   log.debug("Link: " + trend.getLink());
                   log.debug("Plain text description: " + trend.getDescription());
       }
     }
   }
  }
 }

The sequence diagram of the above components interaction is given below.

Google Trends Spring Batch

The batch configuration is given below.

package org.smarttechie;

import java.util.List;

import org.smarttechie.model.GoogleTrendItem;
import org.smarttechie.reader.GoogleTrendsReader;
import org.smarttechie.writer.GoogleTrendsWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableBatchProcessing
public class GoogleTrendsBatchConfiguration {

@Bean
public ItemReader<List<GoogleTrendItem>> reader() {
   GoogleTrendsReader reader = new GoogleTrendsReader();
   return reader;
}

@Bean
public ItemWriter<List<GoogleTrendItem>> writer() {
   GoogleTrendsWriter writer = new GoogleTrendsWriter();
   return writer;
}

@Bean
public Step step1(StepBuilderFactory stepBuilderFactory, ItemReader<List<GoogleTrendItem>> reader,
ItemWriter<List<GoogleTrendItem>> writer) {
    return stepBuilderFactory.get("step1")
    .<List<GoogleTrendItem>, List<GoogleTrendItem>> chunk(1)
    .reader(reader)
    .writer(writer)
    .build();
}

@Bean
public Job importUserJob(JobBuilderFactory jobs, Step s1) {
  return jobs.get("importHotSeachesJob").preventRestart()
  .start(s1)
  .build();
 }
}

The above implementation doesn’t have ItemProcessor which is optional. The source code created as part of this article is available here.

Advertisements

I am Siva Prasad Rao Janapati. Working as a software developer. Has hands on experience on ATG Commerce(DAS/DPS/DCS), Mozu commerce, Broadleaf Commerce, Java, JEE, Spring, Play, JPA, Hibernate, Velocity, JMS, Jboss, Weblogic,Tomcat, Jetty, Apache, Apache Solr, Spring Batch, JQuery, NodeJS, SOAP, REST, MySQL, Oracle, Mongo DB, Memcached, HazelCast, Git, SVN, CVS, Ant, Maven, Gradle, Amazon Web services, Rackspace, Quartz, JMeter, Junit, Open NLP, Facebook Graph,Twitter4J, YouTube Gdata, Bazzarvoice,Yotpo, 4-Tell, Alatest, Shopzilla, Linkshare. I have hands on experience on open sources and commercial technologies.

Tagged with: , ,
Posted in Spring, Spring Batch

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

DZone

DZone MVB

Java Code Geeks
Java Code Geeks
%d bloggers like this: