Skip to main content

ATG Search architectural flow : Search and Index



I would like to explain the high level ATG Search implementation architecture ( for an online store) through the above diagram. In this diagram 1.x denotes the search functionality and 2.x denotes the indexing functionality. I have given JBoss as the application server.

Physical Boxes and Application Servers in the diagram ( as recommended by ATG )  :
  1. Estore ( Commerce ) Box --> The box with the estore/site ear (with the site JSPs and Java codes).
  2. Search Engine Box --> The box with the search engine application running.
  3. Indexing Engine Box --> The box with the indexing engine application running.
  4. CA (Content Administration) Box --> The box with the ATG CA ear ( where we could take CA -BCC - Search Administration and configure the search projects) .
  5. Search Indexer Box --> The box with the ATG Search Index ear ( to fetch the index data from repository). Note that the engine performing indexing will need access to the data it is indexing, which for production is the production repository. It will typically access the data via a commerce instance.  For best performance, and for large repositories, that commerce instance should be dedicated for search indexing, and should be a fast machine.
1. Search functionality flow details :

     1.1   Estore server will find the search engine box's host and search engine application running port details from the Search repository
     1.2   Estore server will call the Search engine application as a SOAP request using this host an port
     1.3   Search engine will find the search results using the index file
     1.4   Search engine will send the search results back to the Estore server

2. Index functionality flow details :

     2.1   CA server will start the indexing and will call the Search Index server to fetch the data to be indexed from the repository
     2.2   Search Index server will fetch the data from the catalog repository
     2.3   CA server will call the Index engine application to create the index files
     2.4   Index engine application will create the index files and keep it in a shared folder so that all the search engine applications can read it
     2.5   During the index deploy phase, all search engines will copy the index files to a local folder for fast access

If you need more details, please comment so that I can answer your specific questions  :-)


Labels:

Comments

  1. Anonymous18 January 2012 at 15:55

    nice article

    ReplyDelete
  2. Anonymous15 February 2012 at 06:14

    it is great and simple. Incase both Ca, index and search engine are on same host even then will it communicate via RMI?

    ReplyDelete
  3. Sony Thomas15 February 2012 at 12:26

    First of all, it is not recommended to have these 2 servers and search engine in the same host due to performance reasons. If we have them on the same box, communication with the search engine will always be through SOAP http request and from CA to search index will be via the remote port defined in search indexer.

    ReplyDelete
  4. Anonymous9 March 2012 at 01:49

    Does the engine query the latest deployed index for the results or does it query the index file that is mentioned as part of the search request?

    If it does the latter, how does the index information get updated on the client?

    ReplyDelete
  5. Sony Thomas9 March 2012 at 10:12

    The engine will look into the current deployed index. The current deployed index can be the latest index or can be an old one deployed through historical index deploy.

    Why the client need the index information ?

    ReplyDelete
  6. Anonymous26 July 2012 at 16:46

    All articles related to ATG are really interesting and easy to understand Thomas! Great work.

    ReplyDelete
  7. Anonymous29 May 2014 at 12:10

    Ur are really amazing SONY.... Iam New to ATG.. I want to know more concepts of ATG will u help me plzzz.....

    ReplyDelete

Post a Comment

Popular posts from this blog

Google Chrome shortcut keys

If you are a Google Chromey guy, please find below the list of shortcut keys for some of the most used features  :-) Find more shortcut keys @  http://www.google.com/support/chrome/bin/static.py?page=guide.cs&guide=25799&topic=28650
4 comments
Read more

ATG Search - search engine tuning settings

In this blog, I am going to list the best tuning settings for ATG Search engine. The AESoapConfig.xml, AESoapWaspConfig.xml  and AEConfig.xml are the xmls referred below and you can find it @  <ATG_DIR>\<Searchx.x>\SearchEngine\<operating_system>\bin\ folder. (1)  Make sure that the AESoapConfig.xml's rwTimeout is less than or equal to routing's readTimeoutMs. You could find the routing's readTimeoutMs @ atg\search\routing\SearchEngineService component.               rwTimeout is the  length of time in seconds to wait before a read or write operation times out on an active connection. The number can be decreased to improve performance. However, a value that is too low could cause slow connections to be prematurely closed. (2)  Adjust the number of engine threads to match the number of CPUs available to the engine. Note that the minimal value for maxThreads and maxSpar...
Post a Comment
Read more

Good features of Eclipse 3.6 (Eclipse Helios) JDT

Read the Eclipse Galileo features @  http://tips4ufromsony.blogspot.com/2011/10/good-features-of-eclipse35-eclipse.html New options in Open Resource dialog : The Open Resource dialog supports three new features: • Path patterns: If the pattern contains a /, the part before the last / is used to match a path in the workspace: • Relative paths: For example, "./T" matches all files starting with T in the folder of the active editor or selection: • Closer items on top: If the pattern matches many files with the same name, the files that are closer to the currently edited or selected resource are shown on top of the matching items list. MarketPlace :  Searching and adding new plugins for Eclipse have always been a challenge. The Eclipse Marketplace makes this much easier – it allows you to not only search a central location of all Eclipse plugins, but also allows you to find the most recent and the most popular plugins. Fix multiple proble...
Post a Comment
Read more

ATG Search Indexing - overview of different steps in search indexing

Read more about the search indexing behind the scene steps @  http://tips4ufromsony.blogspot.in/2011/12/atg-search-indexing-behind-scene-steps.html ATG Search prepares searchable content by indexing the products specified in the XML definition file (/atg/commerce/search/ProductCatalogOutputConfig). Generally there are two types of indexing 1.  Full Indexing  --> all data taken for indexing 2.  Incremental Indexing --> only changed data will be taken for indexing When full indexing is triggered, following happens:    1. The out of box component BulkLoader will call IndexedItemsGroup.getGroupMembers() to load the products to the XHTL document. It prevents uncategorized products from getting indexed. The definition file format begins with a top-level item as a product and includes the properties of parent category and childskus. For each product, the set of Variant Producers configured in ProductCatalogOutputConfig is execute...
2 comments
Read more

ATG - how to use the Transaction Manager

Basic concepts of Java transaction In its simplest definition, a transaction is a set of actions that is treated as an atomic unit; either all actions take place (the transaction commits), or none of them take place (the transaction rolls back). Managing transactions is one of the primary tasks of an application server. The application server keeps track of transactions, remembering which transaction is associated with which request, and what transactional resources (such as JDBC or JMS connection) are involved. Each active transaction is represented by a transaction object, which implements the interface javax.transaction.Transaction. A transaction is usually associated with a thread and only one transaction can be associated with a thread at any one time. A central service, called the Transaction Manager, is responsible for keeping track of all these transactions. The Transaction Manager is implemented through the Java Transaction API (JTA). Transaction and Dynamo ap...
3 comments
Read more