Tips from Sony Thomas

The definition file format begins with a top-level item element that specifies the repository and item descriptor to use, and then lists the properties of that item type to include. The top-level item element has the is-document attribute set to true. This attribute specifies that an XHTML document should be generated for each item of that type (in this case, each user item). Property values that come from standard JavaBean properties of the RepositoryItem object (rather than dynamic bean properties) are specified using a dollar-sign ($) prefix. The item element has an is-multi attribute for specifying multi-value properties. If a property is an Array, Collection or Map  you should set this attribute to true. Eeach ATG Search document is uniquely identified by a URL (typically the path name of the file on the file system). In the XHTML documents that the ATG platform generates from repository items, meta properties are represented by meta tags in the head of the document, whi...

Here I am going to explain the steps involved in configuring the multi-language ATG Search if your site support multiple language. I have given the different steps in indexing and search flows. Indexing flow : You could configure different search projects for different locales to support parallel search indexing for each locale. You could setup search environment for the search projects so that the indexing host is different for the two projects. Configure the LocaleVariantProducer @ ProductCatalogOutputConfig to specify the locale for the new language. Include the language dictionaries you need in Search Admin on the Search Projects which can be used for indexing and searching in multiple languages. Specify the locales @ \atg\search\config\LanguageDimensionService so that you could configure the search configurations for each language. When you create the search configuration tree, specify the "Contents vary by" as Language, to configure the search configurations li...

 When the estore or commerce instances are restarted the routing system service can send shut down requests to stale engines, this can be caused because of any of the following. 1. Some other instance marked the engine as stopped in the DB (This can be caused because the machine could not reach the host running the engine) 2. The eStore instance is using a different search schema that has data about the search engine host and is marked as stopped in that DB. To avoid this overrride \atg\search\routing\RoutingSystemService.properties by setting cleanUpStrayEng=false.

Read more about the basic design decisions for a commerce search setup @  http://tips4ufromsony.blogspot.in/2012/03/basic-design-decisions-for-commerce.html Read more about search engine tuning @  http://tips4ufromsony.blogspot.in/2012/02/atg-search-search-engine-tuning.html Following items will affect search performance:  number of items to be searched, quantity of content for each item, number of meta tags, number of facets, Search Merchandising settings, constraints, number of returned results,... You could find a detailed analysis below : 1. Decide how many search engines you need :   Based on the performance requirement of the ecommerce site, create the Search Project @ BCC - Search Admin and add the search engine hosts. Read more about search project creation and search engine host set up @   http://tips4ufromsony.blogspot.com/2012/01/atg-search-how-to-create-search-project.html . 2. Decide the search indexing properties :   Search i...

In this blog I would like to explain the basic set of configuration/design decisions needed to setup an ATG search project. Most of these design decisions are common for all Enterprise search applications. 1. Decide the searchable properties :   This means the properties that the business want the user to search in the ecommerce platform. In ATG search these are configured as the text properties in the product-catalog-output-config.xml ( the definitionFile of the \atg\commerce\search\ProductCatalogOutputConfig). Usually the displayName of product/sku, displayName of department/category/sub-category, skuId, brandName are the properties configured as searchable. 2. Decide the search refinement properties or the faceted properties :   After a user search for a keyword, search refinement is the next step done to filter his results. ATG supports the search refinement using the Faceted Search concept. Read more about facted search @...

Read more about the search indexing behind the scene steps @  http://tips4ufromsony.blogspot.in/2011/12/atg-search-indexing-behind-scene-steps.html ATG Search prepares searchable content by indexing the products specified in the XML definition file (/atg/commerce/search/ProductCatalogOutputConfig). Generally there are two types of indexing 1.  Full Indexing  --> all data taken for indexing 2.  Incremental Indexing --> only changed data will be taken for indexing When full indexing is triggered, following happens:    1. The out of box component BulkLoader will call IndexedItemsGroup.getGroupMembers() to load the products to the XHTL document. It prevents uncategorized products from getting indexed. The definition file format begins with a top-level item as a product and includes the properties of parent category and childskus. For each product, the set of Variant Producers configured in ProductCatalogOutputConfig is execute...

The comminucation between the Commerce box and the Search engine is through SOAP. Read  more about this architecture @  http://tips4ufromsony.blogspot.in/2011/11/atg-search-architectural-flow-search.html The commerce instance forms the SOAP url just like the below code: private URL getSearchEngineURL(SearchEngine engine) {       SearchEnvironmentHost h =  engine.getSearchEnvironmentHost();       SearchMachine hi = h.getSearchMachine() ;       return new URL( "http://" + hi.getHostname() + ":" + engine.getPort() + "/AEXmlService/" );   } So the commerce instance need the hi.getHostname()  and engine.getPort() to form the url. It is obtained as below: 1. The component / atg/commerce/search/refinement/ CommerceFacetSearchService has the siteName defined, which will be pointing to the environment name defined in the Search Project. Read  more about this search project setup @  http://...

ATG Search configuration rules are specified through the ATG BCC Merchandising UI. Over here you could specify the below set of rules Redirection rules --> If you want to redirect to another page other than the search result page for a search keyword Property Prioritization rules  --> Prioritize certain set of properties and give weightage Result exclusion rules  --> Exclude certain search results Result positioning rules  --> Position / Sort the search result data 1.  To create the search configuration tree, you need to log-in to the ATG BCC Merchandising UI and select the Search Configuration Tree in the Browse tab drop down. Now need to click the Create button and first you need to create the Search Configuration Folder. 2.  When you create the folder, give the name of the folder and select whether the contents vary by Language or Segment. If your ecommerce site need to support more than one language and y...

If the search engine application is running in a separate box than the ATG CA-BCC deployed server, the search engine is invoked through a remoteLauncher running in these boxes. Means, a remoteLauncher needs to run in the host where SearchEngine is installed remotely to start the search engine. You could find the startRemoteLauncher.sh @ ATG/ATG2007.1/Search2007.1/SearchAdmin/bin/startRemoteLauncher.sh Start remoteLauncher using the command   ./startremotelauncher.sh –p <RMI Launcher service Port > & The Launcher Service Port on the host machine can be found in BCC @ Search Administration>Projects > Search Project: gmri_search_en_CA > Environments > Host Machine: > Advanced Settings. If the startRemoteLanucher reports a BindingException , then you need to find the process that is using the launchServicePort. For that run the netstat command like :   netstat -an |grep 10880 Then inorder to identify the process t...

The rules that make up the search configurations are created in ATG Merchandising and stored as repository items in the RefinementRepository. When the product catalog is deployed from the ATG Merchandising environment to the target site, the RefinementRepository is deployed as well. For the soap call from the commerce instance to the search engine, for text queries we normally use refineConfig=”map” and the search engine itself determines the right refine config for the query. When Search Administration initiates indexing of the product catalog, it also invokes the /atg/commerce/search/config/ CatalogRankConfigAdapter component. This component is a customization adapter that manages the process of generating the search configuration files and submitting them to ATG Search. To generate the search configuration files, CatalogRankConfigAdapter invokes the /atg/search/repository/ SearchConfigurationXMLGenerator component (of class atg.repository.search.indexing.config.SearchConfigura...

In this blog, I am going to list the best tuning settings for ATG Search engine. The AESoapConfig.xml, AESoapWaspConfig.xml  and AEConfig.xml are the xmls referred below and you can find it @  <ATG_DIR>\<Searchx.x>\SearchEngine\<operating_system>\bin\ folder. (1)  Make sure that the AESoapConfig.xml's rwTimeout is less than or equal to routing's readTimeoutMs. You could find the routing's readTimeoutMs @ atg\search\routing\SearchEngineService component.               rwTimeout is the  length of time in seconds to wait before a read or write operation times out on an active connection. The number can be decreased to improve performance. However, a value that is too low could cause slow connections to be prematurely closed. (2)  Adjust the number of engine threads to match the number of CPUs available to the engine. Note that the minimal value for maxThreads and maxSpar...

Other than the ProductCatalog repository, ATG search uses the SearchConfigurationRepository, SearchAdminRepository and RefinementRepository. Search Administration requires access to all these 3 repositories. The client application must have access to the routing repository.  Please find below the details of each repository.    1. SearchConfigurationRepository :               #  \atg\search\routing\repository\SearchConfigurationRepository.               #  It stores information about search engines, index structure and deployment information.               #  refers to all “ rout_* "  tables.    2. SearchAdminRepository :                #  \atg\searchadmin\SearchAdminRepository.               #  It stores information about search adminis...

A Facet is a search refinement element that corresponds to a property of a commerce item type. ATG supports the search result refinement using the Faceted Search concept. Read more about facted search @  http://en.wikipedia.org/wiki/Faceted_search . Facet can either be ranges or specific values. Each facet is stored in the RefinementRepository as a separate refineElement repository item. Facets are divided into Global and Local facets. Global facets apply to all the categories and local facets only to the category in which they are created. For example Price/Brand can be considered as the facets that are common for all skus and New Release/Coming Soon can be considered as the facets that are specific to Physical Media products like Vidoe/DVD/Blue-ray/Books. We can use the ATG BCC - Merchandising UI to create facets. The Faceting Property depends on the meta-properties defined in the \atg\commerce\search\product-catalog-output-config.xml ( the def...

In this blog, I have listed some basic ATG Search troubleshooting tips in some general scenarios. 1. If the index did not deploy, consider the following possible causes : Is the DeployShare property configured @ /atg/search/routing/LaunchingService component ?  Is enough space available @ deployment share box for the index ?  Are the RMI ports configured correctly in the RoutingSystemService component ?  If the search engine application is running in a separate box, this application is invoked through a RemoteLauncher running in these boxes. Check whether these RemoteLaunchers are running in these boxes ? 2. If you have trouble in launching one or more search engines, try the following remedies: If the Search engine is standalone, set the /atg/search/routing/LaunchingService component’s engineDir property to the absolute path of the Search engine directory.  3. Search unavailable in the estore page, even when the SearchEngine is in "Running" st...

To get an idea of the SearchEngineActivity logs, please read the blog :  http://tips4ufromsony.blogspot.com/2012/01/atg-search-and-search-engine-activity.html Other than using the SearchEngineActivity logs, we could monitor the search engine request/reponse  by enabling the monitoring in the SearchEngineService component @ /dyn/admin/nucleus/atg/search/routing/SearchEngineService. Once you enable this "monitoringEnabled", each request/response and the response time like last response time and average response time can be found @ SearchEngineService component. After you send a request to the search engine, just refresh the SearchEngineService component and you could see that the total command count will be incremented by one and the response time also get updated. If you click this total command count, a new window will be opened with the request and response details.

Here I am going to explain how we can create a new ATG search project. It involves 3 steps --> Specify the general search project settings, Specify the content of search indexing and Build the index. Below I am elaborating the different steps involved with screen shots : 1. Go to Search Project Administration ui @  BCC and Click the button "New Search Project" to create a new search project. 2. Specify the search project name, give description and click the button "Create Search Project". 3. Click the button "Add Content" to add the search project content. 4. Specify the content name, select the content type and specify the IndexingOutputConfig path if the content type is ATG repository. Specify the remote host and port if you are using another server for fetching the content. 5. Click the content in the left side and expand the advanced option to specify the language and other customizations. 6. Click the ...

We could use the SearchEngineActivity log files to get the request/response to the search engine from a commerce instance. This folder is located in each commerce instance or the instances from which the call to the search engine is done. The SearchEngineActivity log file folder can be configured @ SearchEngineService component ( /dyn/admin/nucleus/atg/search/routing/SearchEngineService). To get the log files for the search engine calls, you need to specify the SearchEngineService.dumpingRequests as true. Then you need to specify the engineActivityPath as the folder in which you need the SearchEngineActivity logs. Below you could find my SearchEngineActivity log folder. Each successful call to the search engine from the commerce instance will create 5 files in the SearchEngineActivity folder : namely  request, response, search engineinfo, stack trace and response row . Each file name start with a specific sequence. You ...

The call from the ATG commerce instance ( Estore instance ) to the search engine is done using the SOAP protocol. Read more about this architecture @  http://tips4ufromsony.blogspot.com/2011/11/atg-search-architectural-flow-search.html . If you have a SOAP UI tool (get it from @  http://www.soapui.org/ ), you could test the request/response from the search engine. You could find the wsdl for this SOAP in the folder in which your ATG search engine is installed. Below screen shot has the deatils of the wsdl : To call the search engine you need to know the port in which the search engine is running. You could get it from the SearchEngineService component of the commerce instance @ /dyn/admin/nucleus/atg/search/routing/SearchEngineService/.  Once you have these details, you could call the search engine with a sample request and a query string to get the response.

Here I would like to explain how your ATG search folder structure looks like after you do the ATG search installation and also about the search engine log files. To get the ATG search, you need to install the ATGSearch<version>Windows.exe after you do the ATG installation. When you install ATG search, the generated folders include an OS dependant folder. In the below screen shot, you could see the "i686-win32-vc71" folder and this is generated since the ATG search ins installed in a windows machine. The ATG search engine is actually an exe file inside this "i686-win32-vc71" folder. Below screen shot give you an idea about the "atgsearch.exe", which is the search engine application. This search engine application (atgsearch.exe) is started by a remote launcher call from the BCC search administration UI.  The log files like "atgsearch_20120106112413_568.log" are the search engine log files where you could see the engine ...

Read more about the search indexing @  http://tips4ufromsony.blogspot.com/2011/11/atg-search-architectural-flow-search.html ATG search indexing involves index file creation, deploying and copying the index file to the search engine's box. The steps can be divided into Initial stage, Preparing Content, Indexing and Deploying. Please find below the detailed analysis of each step. 1. Initial stage:        a. Check whether the folder deployshare configured correctly @ LaunchingService.deployShare  ( \atg\search\routing\LaunchingService.deployShare ). Lets assume that it is configured to \Search2007.1\SearchEngine\i686-win32-vc71\buildedIndexFiles.        b. Lets assume that the index file folder ( \Search2007.1\SearchEngine\i686-win32-vc71\indexFiles)  has the following segments (folders) currently :                     66900...