Skip to main content

ATG Search Indexing - overview of different steps in search indexing


Read more about the search indexing behind the scene steps @ http://tips4ufromsony.blogspot.in/2011/12/atg-search-indexing-behind-scene-steps.html

ATG Search prepares searchable content by indexing the products specified in the XML definition file (/atg/commerce/search/ProductCatalogOutputConfig).

Generally there are two types of indexing
1.  Full Indexing  --> all data taken for indexing
2.  Incremental Indexing --> only changed data will be taken for indexing

When full indexing is triggered, following happens:

   1. The out of box component BulkLoader will call IndexedItemsGroup.getGroupMembers() to load the products to the XHTL document. It prevents uncategorized products from getting indexed. The definition file format begins with a top-level item as a product and includes the properties of parent category and childskus. For each product, the set of Variant Producers configured in ProductCatalogOutputConfig is executed to check how many index items are to be created.

   2. XHTML documents are generated for each product, in order to submit to the engine for indexing. The XHTML is generated based on the definition file specified in ProductCatalogOutputConfig. An XHTML document that represents a Commerce product includes information about its parent category’s properties, as well as information about the properties of the child SKUs.

  3. The definition file, product-catalog-output-config is parsed to generate the text and meta properties, to be added to the index. The Text–properties indicates the properties which can be searched on. The Meta-properties indicate the properties which can be sent as constraints for faceted search. The text property will be specified in <text-properties> tag and meta property in <meta-properties> tag. The properties for which there is a custom property accessor specified, the property accessor is used to obtain the value to be indexed.

  4. After all the products have been added, the out of box PostIndexCustomization is executed to add any refineConfig and rankConfig information. This is used by the engine for generating facets and for manipulating the search results

  In case of failure in indexing, check the following logs
- JBoss server logs - <JBOSS_HOME>\server\atg\logs\server.log
- Dumping request logs Folder - <ATG_HOME>\logs\searchEngineActivity\*.xml ( request and response xmls). These logs will provide whether what was the request send to search engine in xml form and what was the response from engine for a query.
- Soap request logs - <ATG_HOME>\Search2007.1\SearchEngine\i686-win32-vc71\bin. This is used for checking the indexing failing.


Labels:

Comments

  1. Unknown13 June 2013 at 20:40

    Hi, I've been dealing with some questions regarding Search and indexing, I got the process a lot clear now (thanks for that) but I got some questions:
    - if there are problems with facets, like not showing the right facets configured, does that mean there was a problem during the indexing on the PostIndexCustomization?
    - Why during the PostIndexCustomization the indexing can take too long, like 2 hours? And before it took like 30 min, what could be a starting point to find what is wrong?
    - Which one is better, full or incremental indexing? Can both coexist? What do you recommend?

    Thanks a lot!

    ReplyDelete
  2. Sony Thomas5 July 2013 at 21:38

    - If there are issues with facets not showing on the site, first check the data in your database, then the refinement repository, then the refineconfig passed to search engine.

    - I haven't tries incremental indexing.But if you have small changes per day, you can go ahead with incremental, otherwise go with full indexing

    ReplyDelete

Post a Comment

Popular posts from this blog

ATG - more about Forms and Form Handlers

An ATG form is defined by the dsp:form tag, which typically encloses DSP tags that specify form elements, such as dsp:input that provide direct access to Nucleus component properties. Find below a sample dsp:form tag.    <dsp:form action="/testPages/showPersonProperties.jsp" method="post" target="_top">      <p>Name: <dsp:input bean="/samples/Person.name" type="text"/>      <p>Age: <dsp:input bean="/samples/Person.age" type="text" value="30"/>      <p><dsp:input type="submit" bean="/samples/Person.submit"/> value="Click to submit"/>    </dsp:form>   When the user submits the form, the /samples/Person.name property is set to the value entered in the input field.Unlike standard HTML, which requires the name attribute for most input tags; the name attribute is optional for DSP form element tags. If an input tag omits the n...
11 comments
Read more

ATG - basic concepts of ATG

This blog is for the ATG beginners to get some basic overview about ATG. I just given the ATG concepts as a list of numbered points for the ease of understanding. 1. At the framework level, ATG is a               java based application platform for hosting web-based applications, as well as RMI accessible business components,               with an ORM layer,               a component container,               an MVC framework,               and a set of tag libraries for JSP. 2. Art Technology Group(ATG)'s Dynamo Application Server (DAS) is a Java EE compliant application server. DAS is no longer actively developed as ATG recommends using other Java EE applications servers for its products such as BEA WebLogic, JBoss or IBM WebSphere. 3. Prior to ATG 2007, JHTML was used instead of JSP for view purpose. J...
4 comments
Read more

Good features of Eclipse3.5 (Eclipse Galileo) JDT

This blog will list down the new features of Eclipse Galileo JDT. I will write another blog regarding the features of Eclipse Helios and Eclipse Indigo. Read about Eclipse Helios features @  http://tips4ufromsony.blogspot.com/2011/11/good-features-of-eclipse-36-eclipse.html ========================================================== 1. Toggle Breadcrumb —> Will list the name of the file and the method name with respect to your cursor position , on the top of the Eclipse IDE. From here you can go to other methods, other classes in same package , …. Screen shot of Toggle Breadcrumb: ========================================================== 2. From the method call , you can either go to declaration or to implementation Screen shot of implementation call: ========================================================== 3. Advanced Open Type –> You can restrict the open type to a selected Working set only. Screen shot of Advanced Open Type: ==========...
2 comments
Read more

ATG and XML File Combination

              An XML file might appear at several points in the CONFIGPATH. The ATG platform, at runtime,   automatically combines all the XML files in the CONFIGPATH with the same name into a single composite file, which is then used by the appropriate component.               XML files are combined one tag at a time: in other words, tags are matched up, and the combination rules are applied to each pair of matched tags. XML file combination is controlled with an XML attribute named xml-combine. This attribute is used only in the preprocessing stage of XML file combination. The xml-combine attribute can have the following values: xml-combine = "replace" xml-combine = "remove" xml-combine = "append" xml-combine = "append-without-matching" -->  This is the same as  xml-combine="append" , except that embedded tags are not matched and combined recursively. The content is simply appended. ...
Post a Comment
Read more

Mozilla FireFox - how to add security certificate exception urls

If you visit a web site with a secure connection(https) and if the website's security certificate has some problem like the security certificate presented by the website was not issued by a trusted certificate authority  or  the security certificate has expired or is not yet valid, you will get an error page ( like below in IE ) with an option to continue to this website. When this non secure page is loading, in Internet Explorer, you will get an option to specify whether you need to download the page content files like JS, CSS, Images,... But you will not get such an option in FireFox/Chrome and could see only the text data in this new non secure page. If you are doing some local development with FireFox/Chrome and have such a situation, you might want to override this security restriction. FireFox provides some exception url list in secure certificates menu. Go to Options - Advanced - Encryption - View Certificates and click the exception list and...
Post a Comment
Read more