Skip to main content

ATG Search Indexing - behind the scene steps explained

Read more about the search indexing @ http://tips4ufromsony.blogspot.com/2011/11/atg-search-architectural-flow-search.html

ATG search indexing involves index file creation, deploying and copying the index file to the search engine's box. The steps can be divided into Initial stage, Preparing Content, Indexing and Deploying. Please find below the detailed analysis of each step.



1. Initial stage:
       a. Check whether the folder deployshare configured correctly @ LaunchingService.deployShare  ( \atg\search\routing\LaunchingService.deployShare ). Lets assume that it is configured to \Search2007.1\SearchEngine\i686-win32-vc71\buildedIndexFiles.
       b. Lets assume that the index file folder ( \Search2007.1\SearchEngine\i686-win32-vc71\indexFiles)  has the following segments (folders) currently :
                    66900009 @ index engine box
       66900010 @ search engine box
       c. Lets assume that the component SearchEngineService has the "Local Content Path" as following for the search and index environments :
                   Search  environment   --> ../indexFiles/66900010
        Indexing environment  --> ../indexFiles/66900009

2.Preparing Content
     a. start an indexing engine @ step "Load latest pre-index customizations"
     b. delete the folder "\indexFiles\66900009" and create a new folder "\indexFiles\66900011" @ location "SearchEngine\i686-win32-vc71\indexFiles" @ index engine box
     c. copy the files initial.index ( from SearchEngine\i686-win32-vc71\data ) and  LUIStore.stg to “indexFiles\66900011” --> done by indexing engine

3.Indexing
     a. create a new index and stg files @ "SearchEngine\i686-win32-vc71\buildedIndexFiles". First create the index file and then the stg file.
     b. copy the new index and stg files to the folder indexFiles\66900011 --> done by indexing engine
     c. update the SearchEngineService.Local Content Path of the indexing environment to ../indexFiles\66900011
     d. kill the indexing engine

4.Deploying
    a. start a new answer engine
    b. create the folder "indexFiles\66900012" @ answer engine box -->  done by the new engine
    c. copy the new index and stg files to the folder "indexFiles\66900012" -->  done be new engine
    d. update the SearchEngineService.Local Content Path of the search environment to ..\indexFiles\66900012
    e. delete the folder "indexFiles\66900010"
    f.  shutdown the previous running engine

Labels:

Comments

  1. Smarty :)20 April 2012 at 10:55

    Hi Sony, I have a clarification in step 4, It will be great if you can help me understand : Will ATG create a new search engine with a new Port at this phase ? Say we have existing engine running in port 5010, will a new engine be created and assigned 5011 and engine at 5010 will be killed during this deploying step ?

    ReplyDelete
  2. Sony Thomas23 April 2012 at 16:54

    Yes. A new searhc engine will automatically started @ 4.a @ new port. When we create the search project, we can define the range of port numbers for the search engines amd the new port will be opened with in that range.

    :-)

    ReplyDelete
  3. kannan14 August 2012 at 21:51

    Hi Sony,
    First of all, thanks for the post....

    I have some concerns. Why do we need the deploy share ?

    As far as i understood
    at step 3.a index and stg files are created in deployeshare folder.

    at step 3.b the index and stg from deploy share is copied to ndexFiles\66900011.

    at step 4.c the index and stg from deploy share is copied to indexFiles\66900012.

    Please correct me

    ReplyDelete
  4. Sony Thomas26 August 2012 at 09:20

    deploy share is the name of the folder we specify @ /atg/search/routing/LaunchingService. This is where the index files are placed for index deployment. Index deployment is the process when the index files are copied to search instances. so the deploy share will come into picture only @ index deployment.

    ReplyDelete
  5. Unknown17 October 2013 at 18:06

    Hi Sony,

    I am a newbie to ATG..! I am really excited to learn ATG concepts...! I have a doubt in the above topic...what is the need for two segment (folders) you mentioned in the below step and why they are mentioned as Index engine box and search engine box..!

    b. Lets assume that the index file folder ( \Search2007.1\SearchEngine\i686-win32-vc71\indexFiles) has the following segments (folders) currently :
    66900009 @ index engine box
    66900010 @ search engine box

    Also please clarify me why the remote launcher is used..and where en all it should run..?

    ReplyDelete
  6. Amit Kumar30 October 2013 at 15:30

    Hi Sony,

    I have one query i hope you will be helping me in this,
    Is it possible to trigger the index deployment using shell script rather than from administration UI.


    Thanks
    Amit Kumar

    ReplyDelete
  7. Unknown2 May 2014 at 12:02

    Can you give pointers to below error while searching for custoemr in CSC.(ATGProfile env is running)
    Error copying file null\data\initial.index to file ../indexFiles\500001\initial.index: boost::filesystem::copy_file: The system cannot find the path specified: "null\data\initial.index", "..\indexFiles\500001\initial.index"

    ReplyDelete

Post a Comment

Popular posts from this blog

Google Chrome shortcut keys

If you are a Google Chromey guy, please find below the list of shortcut keys for some of the most used features  :-) Find more shortcut keys @  http://www.google.com/support/chrome/bin/static.py?page=guide.cs&guide=25799&topic=28650
4 comments
Read more

Income Tax process and e-filing

http://financeminister.in/income_tax_calculator.php https://incometaxindiaefiling.gov.in Below I am listing the step-by-step activities of the Tax Process that a working professional need to do in a given Financial Year. Here FY refers to Financial Year  and  AY refers to Assesment Year. Each month we will pay the Tax (From APRIL 20xx to MARCH 20xx+1) through our Employer for the FY 20xx – 20xx+1. In the month April 20xx, we will give the investment details to the employer (in our employer specified portal) for the FY 20xx – 20xx+1. In the month January 20xx+1, we will give the investment proof details , Rent receipts… to the Employer Finance Department for the FY 20xx – 20xx+1. In the month MAY/JUNE 20xx+1, employer gives the Form 16 for the FY 20xx – 20xx+1 to us (The proof given by the employer to the employee for the tax paid by the employee). In the month July 20xx+1 (on or before July 31st of every year), we will fill the ITR forms (earlier it was NayaSaral f...
2 comments
Read more

ATG CA - different activity sources used @ BCC

Read about how a new link can be added in BCC home page @  http://tips4ufromsony.blogspot.com/2012/03/atg-ca-bcc-home-screen-how-to-add-new.html Normally an ActivitySource.properties file define the set of actions that it supports under a genericActivityDefinitionFile. But some ActivitySource.properties  define the actions  using the workflowActivityDefinitionFiles. For example consider the default "Content Administration" ,  "SearchAdministration",  " Merchanding "  and "Personalization" options in BCC homepage. Below I listed the ActivitySource.properties and other properties for these links. To get all these activitysource names, just take the / atg/bizui/activity/ActivityManager  component @ dyn/admin. Content Administration ActivitySource  --> /atg/bizui/activity/PublishingActivitySource genericActivityDefinitionFile Search Administration ActivitySource  --> /atg/bizui/...
1 comment
Read more

IFSC and MICR code

For internet banking third party transfer you might need the IFSC or MICR code. IFSC or Indian Financial System Code is an alpha-numeric code that uniquely identifies a bank-branch participating in the NEFT system. MICR or Magnetic Ink Character Recognition is a character recognition technology used primarily by the banking industry to facilitate the processing of cheques. Please find below some useful links to get the IFSC and MICR code for a bank branch: http://rbi.org.in/scripts/neft.aspx http://rbidocs.rbi.org.in/rdocs/content/docs/67440.xls http://bankifsccode.com/ http://banks-india.com/ifsc-code.php http://en.wikipedia.org/wiki/Indian_Financial_System_Code http://en.wikipedia.org/wiki/Magnetic_ink_character_recognition
Post a Comment
Read more

ATG Search architectural flow : Search and Index

I would like to explain the high level ATG Search implementation architecture ( for an online store) through the above diagram. In this diagram 1.x denotes the search functionality and 2.x denotes the indexing functionality. I have given JBoss as the application server. Physical Boxes and Application Servers in the diagram ( as recommended by ATG )  : Estore ( Commerce ) Box --> The box with the estore/site ear (with the site JSPs and Java codes). Search Engine Box --> The box with the search engine application running. Indexing Engine Box --> The box with the indexing engine application running. CA (Content Administration) Box --> The box with the ATG CA ear ( where we could take CA -BCC - Search Administration and configure the search projects) . Search Indexer Box --> The box with the ATG Search Index ear ( to fetch the index data from repository). Note that the engine performing indexing will need access ...
7 comments
Read more