Skip to main content

ATG Search Indexing - overview of different steps in search indexing


Read more about the search indexing behind the scene steps @ http://tips4ufromsony.blogspot.in/2011/12/atg-search-indexing-behind-scene-steps.html

ATG Search prepares searchable content by indexing the products specified in the XML definition file (/atg/commerce/search/ProductCatalogOutputConfig).

Generally there are two types of indexing
1.  Full Indexing  --> all data taken for indexing
2.  Incremental Indexing --> only changed data will be taken for indexing

When full indexing is triggered, following happens:

   1. The out of box component BulkLoader will call IndexedItemsGroup.getGroupMembers() to load the products to the XHTL document. It prevents uncategorized products from getting indexed. The definition file format begins with a top-level item as a product and includes the properties of parent category and childskus. For each product, the set of Variant Producers configured in ProductCatalogOutputConfig is executed to check how many index items are to be created.

   2. XHTML documents are generated for each product, in order to submit to the engine for indexing. The XHTML is generated based on the definition file specified in ProductCatalogOutputConfig. An XHTML document that represents a Commerce product includes information about its parent category’s properties, as well as information about the properties of the child SKUs.

  3. The definition file, product-catalog-output-config is parsed to generate the text and meta properties, to be added to the index. The Text–properties indicates the properties which can be searched on. The Meta-properties indicate the properties which can be sent as constraints for faceted search. The text property will be specified in <text-properties> tag and meta property in <meta-properties> tag. The properties for which there is a custom property accessor specified, the property accessor is used to obtain the value to be indexed.

  4. After all the products have been added, the out of box PostIndexCustomization is executed to add any refineConfig and rankConfig information. This is used by the engine for generating facets and for manipulating the search results

  In case of failure in indexing, check the following logs
- JBoss server logs - <JBOSS_HOME>\server\atg\logs\server.log
- Dumping request logs Folder - <ATG_HOME>\logs\searchEngineActivity\*.xml ( request and response xmls). These logs will provide whether what was the request send to search engine in xml form and what was the response from engine for a query.
- Soap request logs - <ATG_HOME>\Search2007.1\SearchEngine\i686-win32-vc71\bin. This is used for checking the indexing failing.


Labels:

Comments

  1. Unknown13 June 2013 at 20:40

    Hi, I've been dealing with some questions regarding Search and indexing, I got the process a lot clear now (thanks for that) but I got some questions:
    - if there are problems with facets, like not showing the right facets configured, does that mean there was a problem during the indexing on the PostIndexCustomization?
    - Why during the PostIndexCustomization the indexing can take too long, like 2 hours? And before it took like 30 min, what could be a starting point to find what is wrong?
    - Which one is better, full or incremental indexing? Can both coexist? What do you recommend?

    Thanks a lot!

    ReplyDelete
  2. Sony Thomas5 July 2013 at 21:38

    - If there are issues with facets not showing on the site, first check the data in your database, then the refinement repository, then the refineconfig passed to search engine.

    - I haven't tries incremental indexing.But if you have small changes per day, you can go ahead with incremental, otherwise go with full indexing

    ReplyDelete

Post a Comment

Popular posts from this blog

Search Facets - how to create a new search facets in ATG Search

A Facet is a search refinement element that corresponds to a property of a commerce item type. ATG supports the search result refinement using the Faceted Search concept. Read more about facted search @  http://en.wikipedia.org/wiki/Faceted_search . Facet can either be ranges or specific values. Each facet is stored in the RefinementRepository as a separate refineElement repository item. Facets are divided into Global and Local facets. Global facets apply to all the categories and local facets only to the category in which they are created. For example Price/Brand can be considered as the facets that are common for all skus and New Release/Coming Soon can be considered as the facets that are specific to Physical Media products like Vidoe/DVD/Blue-ray/Books. We can use the ATG BCC - Merchandising UI to create facets. The Faceting Property depends on the meta-properties defined in the \atg\commerce\search\product-catalog-output-config.xml ( the def...
4 comments
Read more

ATG User Profile schema ER diagram

Check out the Product Catalog  schema ER-Diagram @  http://tips4ufromsony.blogspot.in/2012/01/atg-product-catalog-schema-er-diagram.html Check out the O rder schema ER-Diagram @   http://tips4ufromsony.blogspot.in/2012/02/atg-order-schema-er-diagram.html If you would like to know the relationship between different User Profile schema tables, please find below screen shot of  Profile schema ER Diagrams.  
Post a Comment
Read more

Jsp and CSS size limits that web developers need to aware

Here I am listing some erroneous cases that might occur in your web development phase, due to some size restrictions. JSP file size limit : You might get some run time exceptions that the JSP file size limit exceeds. Please find below the reason : In JVM the size of a single JAVA method is limited to 64kb. When the jsp file is converted to Servlet, if the jspservice method's size exceeds the 64kb limit, this exception will occur. Keep in mind that this exception depends on the implementation of the JSP translator, means the same JSP code may give an exception in Tomcat and may run successfully in Weblogic due to the the difference in the logic to built the Servlet methods from JSP. The best way to omit this issue is by using dynamic include.For example, if you are using                  <%@ include file="sample.jsp" %> (static include),  replace this to               ...
1 comment
Read more

Eclipse Kepler - good features of Eclipse 4.3 ( Eclipse Kepler ) JDT / Workbench

Detached windows with sash When you detach a view or editor into its own separate window, it now has all the capabilities of a normal workbench window. They now support multiple stacks of views separated by sashes with arbitrary layouts. The detached parts will remain synchronized with the master window that they were detached from. This is especially handy for people developing with two or more monitors, so they can spread views across several monitors and keep them synchronized. Import nested projects The Import Projects wizard now has an option to continue searching for projects to import recursively within any project it finds. This allows you to import physically nested projects at the same time. Open Resource dialog enhancements The Open Resource (Ctrl+Shift+R) dialog now offers direct access to the Show In and Open With menus via drop-down buttons. On platforms that support mnemonics, the buttons are also accessible via Alt+W and Alt+H. Whole word option...
Post a Comment
Read more

Quick Reference to JAVA Servlets

I am writing these quick reference blogs for those who wants to brushup the ideas of each topic. This one will lead you through the basic concepts of JAVA Servlets. How this quick reference guideline is different from the numerous other docs available ?  ,  please read on to get an idea : Servlet Lifecycle : The container will identify the servlet based on the URL Servlet class loading Servlet instantaition Call init method Create a servlet thread for the current request. Before calling service(), will create the request and response objects Call service method Service method will identify whether to call doGet or doPost and call it Call destroy method Different servlet objects : A sinlge servlet instance per JVM ( except for SingleTheadModel) A sinlge HttpSession per web application ( session activation and passivation) A sinlge ServletContext per JVM A sinlge ServletConfig per servlet A sinlge ServletRequest per servlet request Important Servlet rq...
Post a Comment
Read more