Skip to main content

ATG Search Indexing - behind the scene steps explained

Read more about the search indexing @ http://tips4ufromsony.blogspot.com/2011/11/atg-search-architectural-flow-search.html

ATG search indexing involves index file creation, deploying and copying the index file to the search engine's box. The steps can be divided into Initial stage, Preparing Content, Indexing and Deploying. Please find below the detailed analysis of each step.



1. Initial stage:
       a. Check whether the folder deployshare configured correctly @ LaunchingService.deployShare  ( \atg\search\routing\LaunchingService.deployShare ). Lets assume that it is configured to \Search2007.1\SearchEngine\i686-win32-vc71\buildedIndexFiles.
       b. Lets assume that the index file folder ( \Search2007.1\SearchEngine\i686-win32-vc71\indexFiles)  has the following segments (folders) currently :
                    66900009 @ index engine box
       66900010 @ search engine box
       c. Lets assume that the component SearchEngineService has the "Local Content Path" as following for the search and index environments :
                   Search  environment   --> ../indexFiles/66900010
        Indexing environment  --> ../indexFiles/66900009

2.Preparing Content
     a. start an indexing engine @ step "Load latest pre-index customizations"
     b. delete the folder "\indexFiles\66900009" and create a new folder "\indexFiles\66900011" @ location "SearchEngine\i686-win32-vc71\indexFiles" @ index engine box
     c. copy the files initial.index ( from SearchEngine\i686-win32-vc71\data ) and  LUIStore.stg to “indexFiles\66900011” --> done by indexing engine

3.Indexing
     a. create a new index and stg files @ "SearchEngine\i686-win32-vc71\buildedIndexFiles". First create the index file and then the stg file.
     b. copy the new index and stg files to the folder indexFiles\66900011 --> done by indexing engine
     c. update the SearchEngineService.Local Content Path of the indexing environment to ../indexFiles\66900011
     d. kill the indexing engine

4.Deploying
    a. start a new answer engine
    b. create the folder "indexFiles\66900012" @ answer engine box -->  done by the new engine
    c. copy the new index and stg files to the folder "indexFiles\66900012" -->  done be new engine
    d. update the SearchEngineService.Local Content Path of the search environment to ..\indexFiles\66900012
    e. delete the folder "indexFiles\66900010"
    f.  shutdown the previous running engine

Labels:

Comments

  1. Smarty :)20 April 2012 at 10:55

    Hi Sony, I have a clarification in step 4, It will be great if you can help me understand : Will ATG create a new search engine with a new Port at this phase ? Say we have existing engine running in port 5010, will a new engine be created and assigned 5011 and engine at 5010 will be killed during this deploying step ?

    ReplyDelete
  2. Sony Thomas23 April 2012 at 16:54

    Yes. A new searhc engine will automatically started @ 4.a @ new port. When we create the search project, we can define the range of port numbers for the search engines amd the new port will be opened with in that range.

    :-)

    ReplyDelete
  3. kannan14 August 2012 at 21:51

    Hi Sony,
    First of all, thanks for the post....

    I have some concerns. Why do we need the deploy share ?

    As far as i understood
    at step 3.a index and stg files are created in deployeshare folder.

    at step 3.b the index and stg from deploy share is copied to ndexFiles\66900011.

    at step 4.c the index and stg from deploy share is copied to indexFiles\66900012.

    Please correct me

    ReplyDelete
  4. Sony Thomas26 August 2012 at 09:20

    deploy share is the name of the folder we specify @ /atg/search/routing/LaunchingService. This is where the index files are placed for index deployment. Index deployment is the process when the index files are copied to search instances. so the deploy share will come into picture only @ index deployment.

    ReplyDelete
  5. Unknown17 October 2013 at 18:06

    Hi Sony,

    I am a newbie to ATG..! I am really excited to learn ATG concepts...! I have a doubt in the above topic...what is the need for two segment (folders) you mentioned in the below step and why they are mentioned as Index engine box and search engine box..!

    b. Lets assume that the index file folder ( \Search2007.1\SearchEngine\i686-win32-vc71\indexFiles) has the following segments (folders) currently :
    66900009 @ index engine box
    66900010 @ search engine box

    Also please clarify me why the remote launcher is used..and where en all it should run..?

    ReplyDelete
  6. Amit Kumar30 October 2013 at 15:30

    Hi Sony,

    I have one query i hope you will be helping me in this,
    Is it possible to trigger the index deployment using shell script rather than from administration UI.


    Thanks
    Amit Kumar

    ReplyDelete
  7. Unknown2 May 2014 at 12:02

    Can you give pointers to below error while searching for custoemr in CSC.(ATGProfile env is running)
    Error copying file null\data\initial.index to file ../indexFiles\500001\initial.index: boost::filesystem::copy_file: The system cannot find the path specified: "null\data\initial.index", "..\indexFiles\500001\initial.index"

    ReplyDelete

Post a Comment

Popular posts from this blog

ATG - more about Forms and Form Handlers

An ATG form is defined by the dsp:form tag, which typically encloses DSP tags that specify form elements, such as dsp:input that provide direct access to Nucleus component properties. Find below a sample dsp:form tag.    <dsp:form action="/testPages/showPersonProperties.jsp" method="post" target="_top">      <p>Name: <dsp:input bean="/samples/Person.name" type="text"/>      <p>Age: <dsp:input bean="/samples/Person.age" type="text" value="30"/>      <p><dsp:input type="submit" bean="/samples/Person.submit"/> value="Click to submit"/>    </dsp:form>   When the user submits the form, the /samples/Person.name property is set to the value entered in the input field.Unlike standard HTML, which requires the name attribute for most input tags; the name attribute is optional for DSP form element tags. If an input tag omits the n...
11 comments
Read more

Basic design decisions for a commerce search setup ( with an ATG Search view)

In this blog I would like to explain the basic set of configuration/design decisions needed to setup an ATG search project. Most of these design decisions are common for all Enterprise search applications. 1. Decide the searchable properties :   This means the properties that the business want the user to search in the ecommerce platform. In ATG search these are configured as the text properties in the product-catalog-output-config.xml ( the definitionFile of the \atg\commerce\search\ProductCatalogOutputConfig). Usually the displayName of product/sku, displayName of department/category/sub-category, skuId, brandName are the properties configured as searchable. 2. Decide the search refinement properties or the faceted properties :   After a user search for a keyword, search refinement is the next step done to filter his results. ATG supports the search refinement using the Faceted Search concept. Read more about facted search @...
6 comments
Read more

Display date and time for a DATE field @ SqlDeveloper

For date fields, by default SQL Developer will display only the date without time. To set it to display the time as well, do the following: Go to SQL Developer –> Tools >> Preferences. Select Database >> NLS Parameters from the left panel. From the list of NLS parameters, enter DD-MON-RR HH24:MI:SS into the Date Format field. Save and close
Post a Comment
Read more

Lucene, sample JAVA code to Search an indexed file folder

Please find below the Lucene sample JAVA code to search the files inside a folder. This code will search the indexed folder for a search query in an indexed field. This java code is expecting the index path ( where the index files were created ) , field which need to be searched and the query need be searched as program arguments like  "java SearchFiles [-index dir] [-field f] [-query string]" . import java.io.File; import java.util.ArrayList; import org.apache.lucene.analysis.Analyzer; import org.apache.lucene.analysis.standard.StandardAnalyzer; import org.apache.lucene.document.Document; import org.apache.lucene.queryParser.QueryParser; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.Query; import org.apache.lucene.search.ScoreDoc; import org.apache.lucene.search.TopDocs; import org.apache.lucene.store.FSDirectory; import org.apache.lucene.util.Version; public class SearchFiles { public static void main(Stri...
Post a Comment
Read more

GC Log Analyzer from IBM

This blog is about the GC( garbage collection) log analyzer from IBM. If you have a GC log and you want to analyze the file,  this IBM tool will help you with some graphical analyzer and some recommendations. You can download it from the following URL : https://www.ibm.com/support/pages/ibm-pattern-modeling-and-analysis-tool-java-garbage-collector-pmat Please find below some screenshots and details that might help you. 1. Screenshot 1: 2. Screenshot 2 : 3. Screenshot 3 : 4. Screenshot 4:
2 comments
Read more