The customer is using a clustered Netweaver 7.0 SP17 System on Windows IA64/MSSQL2005 Platform for Ticket Sales and Reservation System. Thare are Call Center, Ticket Office and Internet sales channels. Most of the applications are built with Webdynpro for Java.
On one cluster node 3 and on the other 4 server processes are defined.
Problem
On each cluster node, more than once server processes (not all of them at the same time time) keeps restarting during the whole day. This affects the availability and performance of the system to respond user requests.
Also, the other problem was that the logon times are very high on each node.
Solution Approach
When we check the std_server.out log files that are exists under :\usr\sap\\JC or JC\work we noticed lots of OutOfMemory problems for the server processes.
We checked the Java Memory parameters as per note 723909 and noticed that everything is ok.
Then we decided to use the Memory Analyzer Tool to analyze the problem causing the server processes to restart.
Eclipse Memory Analyzer is well defined and explained @ https://www.sdn.sap.com/irj/scn/wiki?path=/display/Java/Java+Memory+Analysis, please check this tool.
With this tool, we able to analyze the thread dumps created as system take and OutOfMemory error.
This is because, we set the following Java Parameter for each Server Processes via ConfigTool.
-XX:+HeapDumpOnOutOfMemoryError
With the dump files, we analyzed and fund the major suspects for the OutOfMemory as:
The classloader/component "service:webdynpro" occupies 856.456.000 (41,55%) bytes. The memory is accumulated in one instance of "java.util.Hashtable$Entry[]" loaded by "".
There are about 851.110.232 Accumulated Objects of java.util.Hashtable$Entry[399]
class com.sap.tc.webdynpro.clientserver.session.SessionManager @0x10a2d86e0
classloader> class com.sap.tc.webdynpro.services.session.WatchDog @0x108931ee8 32 32 com.sap.tc.webdynpro.services.session.WatchDog @ 0xa4163de8 BusyMonitor
This showed us that the problem is caused by the Webdynpro Infrastructure, not by one ouf our application. Also this also showed us that the SessionManager is also a cause and this affects the logon performance.
Solution
For the solution we check lots of resources and found nothing. And then opened an OSS Message. Thanx to Ashutosh from AS Java team, informed us that the problem is caused by the Webdynpro Framework and solved with latest hotfixes for SP17.
But we installed the SP18 for NW Portal and solved the problems of server processes restart and logon performance.
