Auto Complete Suggestions Implementation Using Apache Solr- Approach1

Apache Solr Auto Suggestions Or Auto Completion

                  In this article, we are going to discuss “how to set up auto suggest or auto completion feature by using Apache Solr?“. Before getting into the set up, we will see “Why we require auto suggest feature?“.

                      In today’s internet world, most of the websites are running with data and they are having search feature to find out the content. If the user don’t know, what key word to be used to search for the content, user requires some suggestions from the site. These suggestions will improve the users interactivity. Auto suggestions helps the users to find what they are looking for, by typing the least possible number of characters. The below picture shows how the auto suggestions will be provided in the site.

Auto Suggestions

Now, We will see the auto suggestions feature set up. In the previous articles, we knew the Solr set up. So, here, Straight away  I am going to explain the auto suggest feature configuration. Follow the below given steps.

Step 1:

Set up the data for auto suggestions. The data model is given below.

Data model for auto suggestions or auto completion

                  In the table, TERM column will contain the search key words that the user is used. The VOLUME column contains the number of times the user is used the search key word. Based on this volume, we will come to know, the popular search key words users used. The sample data dump is available here.

                Basically, in the enterprise applications, we require user generated content(the key words which users are using to search) to drive the auto suggestions. Here, we are going to capture all the user entered key words in to a database table or we will find out the used key words for search from the application log files.

Once the data set up is done, we need to configure the Solr’s data config file required for data import handler.

Step 2:

The Solr’s data configuration is given below.

<dataConfig>
 <dataSource driver="" url="" user="" password=""/>
 <document>
   <entity name="autosuggestions" query="SELECT ID, TERM FROM auto_suggestions">
    <field column="ID" name="Id" />
    <field column="TERM" name="term" />
   </entity>
 </document>
</dataConfig>

Step 3:

Configure the schema.xml. In the schema.xml, the analyzer portion is very important. The configuration is given below.


<fieldType name="text_suggest" class="solr.TextField" positionIncrementGap="100">
 <analyzer type="index">
   <tokenizer class="solr.WhitespaceTokenizerFactory"/>
   <filter class="solr.LowerCaseFilterFactory"/>
   <filter class="solr.EdgeNGramFilterFactory" minGramSize="1" maxGramSize="1000"/>
 </analyzer>
 <analyzer type="query">
   <tokenizer class="solr.WhitespaceTokenizerFactory"/>
   <filter class="solr.LowerCaseFilterFactory"/>
 </analyzer>
</fieldType>

The fields configuration is given below.


<fields>
 <field name="id" type="integer" indexed="true" stored="true" required="true"/>
 <field name="term" type="text_suggest" indexed="true" stored="true"/>
</fields>

Step 4: 

Do the full import of the data. Once the full import is done, perform some searches to get the auto suggestions.

For example , try to get the auto suggestions for key word “lap“,  http://:/solr/autoSuggestApproach1/select?q=term%3A%22lap%22&wt=xml&indent=true&rows=10

The response is given below.


<?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
 <int name="status">0</int>
 <int name="QTime">1</int>
 <lst name="params">
 <str name="indent">true
 <str name="q">term:"lap"
 <str name="wt">xml
 <str name="rows">10
 </lst>
</lst>
<result name="response" numFound="376" start="0">
 <doc>
 <str name="term">laptops
 <int name="id">5510</int></doc>
 <doc>
 <str name="term">laptop 15r
 <int name="id">5168</int></doc>
 <doc>
 <str name="term">laptop 199
 <int name="id">5173</int></doc>
 <doc>
 <str name="term">laptop 2gb</str>
 <int name="id">5175</int></doc>
 <doc>
 <str name="term">laptop 400</str>
 <int name="id">5176</int></doc>
 <doc>
 <str name="term">laptop 5i</str>
 <int name="id">5177</int></doc>
 <doc>
 <str name="term">laptop a</str>
 <int name="id">5180</int></doc>
 <doc>
 <str name="term">laptop dv7</str>
 <int name="id">5365</int></doc>
 <doc>
 <str name="term">laptop gsm</str>
 <int name="id">5384</int></doc>
 <doc>
 <str name="term">lap tray laptop</str>
 <int name="id">5152</int></doc>
</result>
</response>

                     The above response is having the auto suggestions starting with “lap” . This response , we  can use to present the auto suggestions/ auto completion feature in the site.

                     The  Solr set up used in this article is available here. The set up which we have done in this article, will provide the auto suggestions starting with the search key word. In the coming article, we will discuss, the configuration to provide the auto suggestions matching with the search key word any where.

Advertisements

I am Siva Prasad Rao Janapati. Working as a software developer. Has hands on experience on ATG Commerce(DAS/DPS/DCS), Mozu commerce, Broadleaf Commerce, Java, JEE, Spring, Play, JPA, Hibernate, Velocity, JMS, Jboss, Weblogic,Tomcat, Jetty, Apache, Apache Solr, Spring Batch, JQuery, NodeJS, SOAP, REST, MySQL, Oracle, Mongo DB, Memcached, HazelCast, Git, SVN, CVS, Ant, Maven, Gradle, Amazon Web services, Rackspace, Quartz, JMeter, Junit, Open NLP, Facebook Graph,Twitter4J, YouTube Gdata, Bazzarvoice,Yotpo, 4-Tell, Alatest, Shopzilla, Linkshare. I have hands on experience on open sources and commercial technologies.

Tagged with: ,
Posted in Apache Solr, Auto Suggestions
5 comments on “Auto Complete Suggestions Implementation Using Apache Solr- Approach1
  1. Anonymous says:

    I tried to implement as your suggestion but the result is not good.
    The system informed :
    org.apache.solr.common.SolrException: SolrCore ‘search’ is not available due to init failure: RequestHandler init failure
    org.apache.solr.common.SolrException: SolrCore ‘search’ is not available due to init failure: RequestHandler init failure
    at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:745)
    at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:110)
    at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
    at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
    at com.homedirect.moneydirect.search.DataSearch.top(DataSearch.java:257)

    I don’t know exactly about this. Can you know the way to solve this ?
    Thanks.

  2. Lasha says:

    Great blog, continue the good work!

  3. Liam says:

    Your web site provided us with valuable information to work on. You’ve
    done an impressive job and our entire community will be grateful to you.

  4. Kacey says:

    Good explanation and informative steps to implement the auto suggetsions. Thank you

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

DZone

DZone MVB

Java Code Geeks
Java Code Geeks
%d bloggers like this: