Thursday, 26 January 2012

CamStudio --> To capture your screen activity into video (Screen casting free software)

CamStudio is a tool (open source) for recording screen activity into standard AVI video files (screen casting software). It also have the audio record feature. It can also used to convert AVIs into Flash Video format. Read more about screencast @ You can download CamStudio from:   or .

I have uploaded a demo video, recorded using the Camstudio release 2.6. You could watch a high quality  video @ Youtube:

or you could find a video with low resolution below :

CamStudio can be used to:
  1. Create movies used in user trainings
  2. Demonstrate features of a new software
  3. Track the progress of a program that executes for a long time
  4. Record the sequence of steps that cause the occurrence of bugs in a faulty software
  5. Record a movie stream 
  6. Convert AVI files to Flash (SWF)  files.
You have different options in CamStudio as follows

Setting Regions : You can select the screen region to capture. 3 options available.    
  • Region:  Before recording starts, you can define an arbitrary rectangular region with the mouse.
  • Fixed Region:  You can capture a fixed-size rectangular region. A pop up allows you to define the size of the region in pixels.
  • Screen: The entire computer screen is captured.
  • Window: To capture a window ( added in 2.6 release)

Setting Video Options: Different options like Quality, Compressor, Capture Frames,...

Setting Audio Options: Different options like No audio, Audio from Speakers, Audio from Microphone,...

Setting Cursor Options: Different options like Hide Cursor,Show Cursor,...

Program Options: Different options like Name of AVI, Directory for recording, Play AVI file when recording stops, Minimize program on start recording ,...

Screen Annotations: The Screen Annotations feature allows you to add captions to your screen during recording.

Producing Flash (SWF) Files: CamStudio allows you to produce Flash files in two ways Record directly to Flash  or Convert an existing AVI file to Flash with the SWF Producer

To compare the different screen casting software,  please check the wiki link :

Wednesday, 25 January 2012

ATG search and Monitoring the request and response

To get an idea of the SearchEngineActivity logs, please read the blog :

Other than using the SearchEngineActivity logs, we could monitor the search engine request/reponse  by enabling the monitoring in the SearchEngineService component @ /dyn/admin/nucleus/atg/search/routing/SearchEngineService.

Once you enable this "monitoringEnabled", each request/response and the response time like last response time and average response time can be found @ SearchEngineService component. After you send a request to the search engine, just refresh the SearchEngineService component and you could see that the total command count will be incremented by one and the response time also get updated.

If you click this total command count, a new window will be opened with the request and response details.

Tuesday, 24 January 2012

ATG Search --> How to create a search project

Here I am going to explain how we can create a new ATG search project.

It involves 3 steps --> Specify the general search project settings, Specify the content of search indexing and Build the index.

Below I am elaborating the different steps involved with screen shots :

1. Go to Search Project Administration ui @  BCC and Click the button "New Search Project" to create a new search project.

2. Specify the search project name, give description and click the button "Create Search Project".

3. Click the button "Add Content" to add the search project content.

4. Specify the content name, select the content type and specify the IndexingOutputConfig path if the content type is ATG repository. Specify the remote host and port if you are using another server for fetching the content.

5. Click the content in the left side and expand the advanced option to specify the language and other customizations.

6. Click the environments in the left side and you could see the environments.You can add/remove a new Production/Staging environment.

7. Click on the Environment name to specify the host where you want to run the search engine. Here you could add more search engine hosts.

8. After you specify all these details, click on the build index button to build the search index and to complete the search project creation.

Friday, 20 January 2012

ATG Search and Search engine activity log

We could use the SearchEngineActivity log files to get the request/response to the search engine from a commerce instance. This folder is located in each commerce instance or the instances from which the call to the search engine is done. The SearchEngineActivity log file folder can be configured @ SearchEngineService component ( /dyn/admin/nucleus/atg/search/routing/SearchEngineService).

To get the log files for the search engine calls, you need to specify the SearchEngineService.dumpingRequests as true. Then you need to specify the engineActivityPath as the folder in which you need the SearchEngineActivity logs.

Below you could find my SearchEngineActivity log folder.

Each successful call to the search engine from the commerce instance will create 5 files in the SearchEngineActivity folder : namely request, response, search engineinfo, stack trace and response row.

Each file name start with a specific sequence. You could see that the file name like "2630436491482_5900002_D-179430_6072_search_request.xml"  has first part "2630436491482" a sequence number for each request to the search engine,  then "5900002" is the search engine index file logical partition id, then "D-179430_6072" is the search engine host machine name and port in which the search engine listening.

The  request and response xmls are the SOAP request and response data from the search engine. Below given a sample request xml screen shot.

The engine details log file has the search engine info, like to which search engine the call is done and the port of the engine and the elapsed time,...Below given a sample search engine info xml screen shot.

The response raw is the response xml in a raw format and the stack trace has the full trace of the class and methods called for a single search engine call.

If the call to the search engine is unsuccessful or has got some error, then the log file name will have that specific deatils. For example the file name "4739687885869_6100002_D-179430_6074_search_timeout_request.xml" indicates that there was a search engine time out occurred for this call to the search engine.

Wednesday, 18 January 2012

ATG Product Catalog schema ER diagram

Check out the Order schema ER-Diagram @

Check out the User Profile schema ER-Diagram @

If you would like to know the relationship between different Product Catalog tables, please find below screen shots of  Product Catalog schema ER Diagrams.

Tuesday, 17 January 2012

How can we use SOAP UI to test ATG search

The call from the ATG commerce instance ( Estore instance ) to the search engine is done using the SOAP protocol. Read more about this architecture @ If you have a SOAP UI tool (get it from @, you could test the request/response from the search engine.

You could find the wsdl for this SOAP in the folder in which your ATG search engine is installed. Below screen shot has the deatils of the wsdl :

To call the search engine you need to know the port in which the search engine is running. You could get it from the SearchEngineService component of the commerce instance @ /dyn/admin/nucleus/atg/search/routing/SearchEngineService/. 

Once you have these details, you could call the search engine with a sample request and a query string to get the response.

Friday, 13 January 2012

ATG search installation and search engine log files

Here I would like to explain how your ATG search folder structure looks like after you do the ATG search installation and also about the search engine log files.

To get the ATG search, you need to install the ATGSearch<version>Windows.exe after you do the ATG installation. When you install ATG search, the generated folders include an OS dependant folder. In the below screen shot, you could see the "i686-win32-vc71" folder and this is generated since the ATG search ins installed in a windows machine.

The ATG search engine is actually an exe file inside this "i686-win32-vc71" folder. Below screen shot give you an idea about the "atgsearch.exe", which is the search engine application. This search engine application (atgsearch.exe) is started by a remote launcher call from the BCC search administration UI. 

The log files like "atgsearch_20120106112413_568.log" are the search engine log files where you could see the engine logs like the index file used by the search engine,enigne load ,enigne throughput and query response time,.. You could enable the debug logging for the search engine to find the different request processing by the engine including the ip address of the requesting host. Also the search engine start and end time can be obtained from this log.

The engine log will show slow request based on the <SlowRequestThreshold> parameter in the AEConfig.xml, like <SlowRequestThreshold>5000</SlowRequestThreshold>. If you set engineLogLevel to D in launchingService component, you will be able to see the ip address of  of the requesting host.

                 engineLogLevel = D

Example of the search engine log with requesting host ip:
2011/03/22 11:12:59 : DEBUG : Server : 0 : Accept socket 12068 at port 4790 from IP
2011/03/22 11:12:59 : DEBUG : Server : 0 : AEXmlWorker::Shutdown
2011/03/22 11:12:59 : INFO : Server : 0 : Shutdown received
2011/03/22 11:12:59 : DEBUG : Server : 0 : Accept socket 12068 at port 4791 from IP
2011/03/22 11:12:59 : INFO : Server : 0 : Waiting for pending requests to complete.
2011/03/22 11:12:59 : INFO : Server : 0 : Exiting engine
2011/03/22 11:12:59 : INFO : Server : 0 : Scheduled shutdown.

Consider the following search engine log :

2011/03/11 09:36:35 : INFO : Server : 0 : load: 0.6875 throughput: 1.72804 q/s time: 6.60434 s

Load  of .6 means that 60% of the search engine thread are active. Throughput is the number of requests that the engine is able to server per second. Time is the average response time.

So load is specified as percentage and 0.6875 is roughly 69 percent of load, where as anything greater then 1 would suggest the search engine is overburden. Throughput is telling us how many requests the engines can handle across all the cores of the engine hosted box. So throughput is kind of like saying how many request can be placed into the bucket. But it doesn't say how fast the objects in the bucket are moving.

Thursday, 12 January 2012

Useful windows shortcut keys

Following are the most useful set of windows shortcut keys that I would like to share with you.

run commands
  1. calc        -->  Calculator 
  2. mspaint   -->  To get Paint
  3. textpad   -->  To get Textpad if installed
  4. cmd        -->  Opens a new Command Window (cmd.exe) 
  5. control    -->  Displays Control Panel 
  6. msconfig   -->  Configuration to edit startup files 
  7. compmgmt.msc  -->  Computer management 
  8. fsmgmt.msc      -->  Folder Sharing Management 
  9. services.msc     -->   System Services 
  10. appwiz.cpl  --> Add/Remove Programs 
  11. ncpa.cpl     -->  Network Connections 
  12. %userprofile%  -->  Opens you User's Profile 
  13. %temp%  -->  Opens temporary file Folder 
  14. temp        -->  Opens another temporary file Folder 
cmd commands
  1. hostname   -->  Returns Computer's name 
  2. ipconfig /all   -->  Shows detailed information for all all network adapters
  3. netstat -a   -->  Displays all active network connections with port
  4. netstat -b   -->  Displays the executables and port numbers, involved in creating each connection like iexplore.exe , chrome.exe , OUTLOOK.EXE, ...
  5. systeminfo  -->  Displays various system information like OS, Processor, System Up Time, Original Install Date, BIOS Version, Physical Memory , ...
microsoft natural keyboard shortcuts
  1. Windows Logo + BREAK -->  Display the System Properties dialog box
  2. Windows Logo + D  -->  Display the desktop
  3. Windows Logo + M  --> Minimize all of the windows
  4. Windows Logo + SHIFT + M  -->  Restore the minimized windows
  5. Windows Logo + E  -->  Open My Computer
  6. Windows Logo + F  -->  Search for a file or a folder
  7. Windows Logo + L  -->  Lock the keyboard
  8. Windows Logo + R  -->  Open the Run dialog box
general keyboard shortcuts
  1. NUM LOCK+Asterisk sign (*) in Windows Explorer   -->  Display all of the subfolders that are under the selected folder
  2. SHIFT when you insert a CD-ROM into the CD-ROM drive   -->  Prevent the CD-ROM from automatically playing 
  3. F2 key  -->   Rename the selected item
  4. F6 key  -->   Cycle through the screen elements in a window or on the desktop
  5. SHIFT+F10   -->  Display the shortcut menu for the selected item

Tuesday, 10 January 2012

Quick Reference to JAVA Servlets

I am writing these quick reference blogs for those who wants to brushup the ideas of each topic. This one will lead you through the basic concepts of JAVA Servlets.

How this quick reference guideline is different from the numerous other docs available ?  ,  please read on to get an idea :

Servlet Lifecycle :
  1. The container will identify the servlet based on the URL
  2. Servlet class loading
  3. Servlet instantaition
  4. Call init method
  5. Create a servlet thread for the current request.
  6. Before calling service(), will create the request and response objects
  7. Call service method
  8. Service method will identify whether to call doGet or doPost and call it
  9. Call destroy method

Different servlet objects :
  1. A sinlge servlet instance per JVM ( except for SingleTheadModel)
  2. A sinlge HttpSession per web application ( session activation and passivation)
  3. A sinlge ServletContext per JVM
  4. A sinlge ServletConfig per servlet
  5. A sinlge ServletRequest per servlet request

Important Servlet rq and rs methods :

      <-- javax.servlet.GenericServlet
                <-- javax.servlet.HttpServlet

        <-- javax.servlet.HttpServletRequest<<Interface>>

getParameter      getCookies        getRequestDispatcher
getParameterValues         getHeader             getSession           getMethod
getParameterNames               getIntHeader           getAttribute
getRemotePort --> remote to servlet is client. So this is client port.
getServerPort --> port to which request send ( eg:8080)
getLocalPort --> port for each thread to handle request

      <-- javax.servlet.HttpServletResponse<<Interface>

setContentType            sendRedirect             setHeader
addCookie      encodeURL         addHeader
getWriter      -->   PrintWriter  --> .print method
getOutputStream --> ServletOutputStream --> .write method

Simple Servlet class to get one parameter and a colection of parameters :

public class ServletOne extends HttpServlet{
    public void doGet(HttpServletRequest request,HttpServletResponse response)
                                                                       throws IOException, ServletException {
            PrintWriter out = response.getWriter();
            String userName = request.getParameter("userName");
            String[] paramValues = request.getParameterValues("userNames");
            Enumeration parameterNames = request.getParameterNames();

General purpose of Servlet methods and objects :

public void init() --> usually to handle the ServletConfig.
public void destroy () --> commonly used for activities like close DB connections opened by the servlet.
ServletContext.InitParameter --> like database lookup name, EJB bean JNDI names
ServletContext.attribute --> like databaseConnection, concurrentUsers
HttpSession.attribute --> like shopping cart

Sample web.xml :




         <url-pattern>/Serv1</url-pattern> --> possible to use wildcards





Servlet and RequestDispatcher :

RequestDispatcher rd = request.getRequestDispatcher("FirstJSP.jsp");   --> Relative or absolute path


RequestDispatcher rd = getServletContext().getRequestDispatcher("FirstJSP.jsp");  --> Absolute path

  • When we use "rd.include", control is retained by the calling resource.
  • When we use "rd.include", not full HttpServletResponse functionality is passed to the called resource.
  • Remember that you cannot forward the request if you have already committed a reponse like "out.print" or "os.flush"
  • To pass some data to JSP you can use any of HttpServletRequest or HttpSession or ServletContext objects

Servlet and HttpSession

Http is a stateless protocol. To get session we can use javax.servlet.http.HttpSession
The HttpSession interface is used with cookie and URL rewriting technology.

header send in response --> Set-Cookie:JSESSIONID=0AABB2345ED2
header send in request --> Cookie:JSESSIONID=0AABB2345ED2
  •    HttpSession session = request.getSesion();
  •    session.isNew()
  •    request.getSession(false) --> only already created session
  •    session.getId();
  •    session.getCreationTime();
  •    session.getLastAccessedTime();
  •    String userName = (String)session.getAttribute("userName");
  •    session.setAttribute("userName",request.getParameter("userName")); --> Cannot store primitive data types
  •    session.setMaxInactiveInterval(int interval);
  •    int interval = session.getMaxInactiveInterval();
  •    session.invalidate();
    <session-timeout>-1</session-timeout> --> will never expire

response.encodeURL("/"); --> URL rewriting -->possible only with dynamically generated urls

Cookie cookie = new Cookie(userName,password);

Cookie[] cookies = request.getCookies();
for(Cookie cookie:cookies){
     String cookieName = cookie.getName();
     String cookieValue = cookie.getValue();

Servlet and Thread safety

Web components share information with other web components using objects that are maintained as attributes of four types of scope object
  • web context --> ServletContext
  • session --> HttpSession
  • request --> ServletRequest
  • page --> PageContext

1. You can ensure that each request works independently and doesn't access the servlet instance concurrently by implementing SingleThreadModel

      public class myServlet extends HttpServlet implements SingleThreadModel
      <%@ page isThreadSafe="true" %> --> The default value of the isThreadSafe attribute is "true".
                                                               --> Will not implement SingleThreadModel.

2. You can allow concurrent access while protecting the shared data in the same way as you do in other multithreaded Java applications. For high-traffic servlets, you should use explicit synchronized blocks rather than implementing the SingleThreadModel interface like
      synchronized(getServletContext()){ ... } or
      synchronized(session){ ... }

Only local variables and request parameters are thread safe.

Servlet and Filter

Filters are commonly used for :
  • authentication
  • localization
  • encryption
  • data compression
  • caching
  • logging --> given below
javax.servlet.Filter ---> init(), doFilter(), destroy()

      public void init(FilterConfig config)
      public void doFilter(ServletRequest request, --> Please note that this is not HttpServletRequest
                                    ServletResponse response,
                                    FilterChain chain) throws IOException,ServletException






       <dispatcher>REQUEST</dispatcher> or INCLUDE or FORWARD or ERROR
                     --> from 2.4 version onwards   --> REQUEST is the default value --> All/any of thesre can be included

Container's rule for ordering the filters:
        1. All filters with "<url-pattern>" is taken in the order in which they are declared in DD.
        2. All filters with "<servlet-name>" is taken in the order in which they are declared in DD.

FilterChain.doFilter() --> Will invoke the next filter.
String filterName = filterConfig.getFilterName();

Logging using the filter:

     ServletContext context = filterConfig.getServletContext();
     context.log(new Date() + ": " + "Access to: " + req.getRequestURL()
                       + " requested by: " + req.getRemoteHost()
                       + ". (Reported by " + filterConfig.getFilterName() + ".)");

Compression using the filter:
     It can be used after the FilterChain.doFilter() call
        public void doFilter(request,response,chain) {
                     //request handling
                     //response handling --> compression.

     Issue --> After the call "chain.doFilter()", response will already be flushed to client from servlet.
     Solution --> Create a new instance of "response" and copy all values to it and pass it.
             Issue --> The "response" has n values to copy --> need to code all ?
             Solution --> use Wrappers

                              ServletRequestWrapper               ServletResponseWrapper
                              HttpServletRequestWrapper        HttpServletResponseWrappers

Servlet and Listener

Use of Listener:   If you wanted to keep a count of the total number of users on a system, you could update a global userNumber variable upon receiving each session creation event(sessionCreated). Similarly contextInitialized can be used to create a connection common to all the servlets.


  Changes that trigger events -->
          1. Initialization and destruction of servletContext.
          2. Initialization and destruction of ServletRequest.
          3. Initialization and destruction of session objects.
          4. Addition/Alteration/Removal of attributes associated with servletContext, ServletRequest or session objects.

java.util.EventListener                                         ---> java.util.EventObject

javax.servlet.ServletContextListener                    ---> javax.servlet.ServletContextEvent
contextInitialized getServletContext

javax.servlet.ServletContextAttributeListener      ---> javax.servlet.ServletContextAttributeEvent
attributeAdded                                                           getName
attributeRemoved                                                       getValue --> OLD value
attributeReplaced                                                       getServletContext

javax.servlet.ServletRequestListener                   ---> javax.servlet.ServletRequestEvent
requestInitialized                                                        getServletContext

javax.servlet.ServletRequestAttributeListener      ---> javax.servlet.ServletRequestAttributeEvent
attributeAdded                                                           getName
attributeRemoved                                                       getValue
attributeReplaced                                                       getServletContext

javax.servlet.HttpSessionListener                        ---> javax.servlet.HttpSessionEvent
sessionCreated                                                           getSession

javax.servlet.HttpSessionAttributeListener          ---> javax.servlet.HttpSessionBindingEvent
attributeAdded                                                          getName
attributeRemoved                                                      getValue --> OLD value
attributeReplaced                                                      getSession

javax.servlet.HttpSessionBindingListener            ---> javax.servlet.HttpSessionBindingEvent
valueBound                                                               getName
valueUnbound                                                           getValue
(no need to configure this in DD)                                getSession

javax.servlet.HttpSessionActivationListener       --->  javax.servlet.HttpSessionEvent
sessionDidActivate                                                    getSession

HttpSessionBindingEvent extends HttpSessionEvent

Simpe Tips in servlet :
  1. WEB-INF --> The folder name should be in capital letter to work with Tomcat5.
  2. <form action="/urlOne" > --> If you have a folder like "Tomcat 5.5\webapps\test\urlOne", this will not work as this is an absolute path, this will search for "webapps/urlOne" and you have "webapps/test/urlOne". So need to use <form action="urlOne"> to get "webapps/test/urlOne".
  3. You can overrride the init() method and do the one time operations like create a DB connection.
  4. War files make it easy to transfer the web applications from one server to another.