Monitoring

Analyzing Garbage Collection Log

Hi,
Jay SenSharma

Jay SenSharma

It’s always best to enable the Garbage collection Logging in our production environment as well because it does not cause any resource overhead or any side effect on weblogic server or an other application server’s performance.  GC log helps us in investigating man issues. Apart from issues it helps us to find out if some tuning is required based on the statistics of the Garbage collection.
.
Garbage collection logging can be enable and collected in a separate log file by using the following JAVA_OPTIONS:
-Xloggc:D:/gcLogs/GCLogs.log         -XX:+PrintGCDetails        -XX:+PrintGCTimeStamps
As soon as you add these JAVA_OPTIONS which are JVM specific (above will work for Sun and Open JDKs fine) the JVM will start generating the garbage collection logging in the GCLog.log file. Now if you will open this file then you can
see something like following:
4.636: [GC [PSYoungGen: 230400K->19135K(268800K)] 230400K->19135K(2058752K), 0.0635710 secs] [Times: user=0.08 sys=0.01, real=0.06 secs]
7.302: [GC [PSYoungGen: 249535K->38396K(268800K)] 249535K->51158K(2058752K), 0.0777300 secs] [Times: user=0.21 sys=0.04, real=0.07 secs]
7.521: [GC [PSYoungGen: 49735K->38388K(268800K)] 62496K->51933K(2058752K), 0.0741680 secs] [Times: user=0.15 sys=0.04, real=0.07 secs]
7.595: [Full GC (System) [PSYoungGen: 38388K->0K(268800K)] [PSOldGen: 13545K->51794K(1789952K)] 51933K->51794K(2058752K) [PSPermGen: 19868K->19868K(39936K)], 0.3066610 secs] [Times: user=0.28 sys=0.02, real=0.31 secs]
9.752: [GC [PSYoungGen: 230400K->26206K(268800K)] 282194K->78000K(2058752K), 0.0728380 secs] [Times: user=0.15 sys=0.00, real=0.08 secs]
11.906: [GC [PSYoungGen: 256606K->38393K(268800K)] 308400K->94759K(2058752K), 0.1058920 secs] [Times: user=0.19 sys=0.00, real=0.10 secs]
13.480: [GC [PSYoungGen: 268793K->38394K(268800K)] 325159K->109054K(2058752K), 0.0762360 secs] [Times: user=0.20 sys=0.03, real=0.08 secs]
18.115: [GC [PSYoungGen: 268794K->38384K(268800K)] 339454K->179238K(2058752K), 0.1351350 secs] [Times: user=0.42 sys=0.10, real=0.14 secs]
20.860: [GC [PSYoungGen: 268784K->38394K(268800K)] 409638K->200343K(2058752K), 0.1063430 secs] [Times: user=0.29 sys=0.03, real=0.11 secs]
22.148: [GC [PSYoungGen: 268794K->38399K(268800K)] 430743K->221395K(2058752K), 0.1173980 secs] [Times: user=0.24 sys=0.02, real=0.12 secs]
23.357: [GC [PSYoungGen: 268799K->26775K(268800K)] 451795K->231618K(2058752K), 0.0714130 secs] [Times: user=0.15 sys=0.03, real=0.08 secs]
24.449: [GC [PSYoungGen: 257175K->29170K(268800K)] 462018K->239909K(2058752K), 0.0312400 secs] [Times: user=0.06 sys=0.01, real=0.04 secs]
You can notice something in the above output:
Point-1). [Full GC (System) [PSYoungGen: 38388K->0K(268800K)]    It means a Full GC is happening on the complete Heap Area including all the Areas of the Java Heap Space.
.
Point-2). [GC [PSYoungGen: 230400K->19135K(268800K)]   Indicates some small GCs which keep on happening in the young generation very frequently,This garbage collection cleans the Young Generation short living Objects.
.
Point-3). Meaning of the [GC [PSYoungGen: 230400K->19135K(268800K)]   line is around 256MB (268800K) is the Young Generation Size, Before Garbage Collection in young generation the heap utilization in Young Generation area was around  255MB (230400K)  and after garbage collection it reduced up to 18MB (19135K)
.
Point-4). Same thing we can see for Full Garbage collection as well….How effective the Garbage collection was…[Full GC (System) [PSYoungGen: 38388K->0K(268800K)] [PSOldGen: 13545K->51794K(1789952K)]  Here it says that around
[(old)1789952K +  young (268800K) ]  memory space means  OldGeneration is consuming 1.75GB space and Young Generation is consuming around 255 MB space  So it means total Heap size is around 2GB.
.
But analyzing the Garbage collection log like above technique Line by Line is very bad…so here we have an alternative was to analyze the Garbage Collection log in few Seconds to see how much time the Full Garbage collection is taking as an average and other reports…etc.
.
Step1). Download the “garbagecat-1.0.0.jar   (881 KB) ”  tool from the follwing link: http://garbagecat.eclipselabs.org.codespot.com/files/garbagecat-1.0.0.jar
.
Step2). Open a command prompt and then make sure that JAVA is set in the Path so that we can use “jar” utility of JDK to run the “garbagecat-1.0.0.jar”  tool.
.
Step3). Put the “garbagecat-1.0.0.jar”  file and the “GCLog.log” file in the same directory. then run the following command:
java      -jar      garbagecat-1.0.0.jar      GCLog.log
.
Step4). As soon as ou run the above command you will see that in your current directory following files are created:
garbagecat-1.0.0.jar
GCLog.log
gcdb.lck
gcdb.log
gcdb.properties
report.txt
.
Step5). Now open the “report.txt” file to see the Over all report of the Garbage Collection something like following:
========================================
SUMMARY:
========================================
# GC Events: 12
GC Event Types: PARALLEL_SCAVENGE, PARALLEL_SERIAL_OLD
Max Heap Space: 2058752K
Max Heap Occupancy: 462018K
Max Perm Space: 39936K
Max Perm Occupancy: 19868K
Throughput: 95%
Max Pause: 306 ms
Total Pause: 1233 ms
First Timestamp: 4636 ms
Last Timestamp: 24449 ms
========================================
.
If you see that the Garbage Collection Max Pause time is very high like  more than 5-7 Seconds for a 2 GB heap then you need to worry about it. 😉
NOTE: Garbagecat  is a best utility to generate the Garbage Collection Report for Sun JDK and Open JDK for other JDKs you should use other tools for accurate results.
.
.
Thanks
Jay SenSharma

WebLogic JMS Monitoring Using WLST

Hi,

Jay SenSharma

Jay SenSharma

Many times we want to monitor the different modules available as part of our WebLogic Domain. WebLogic Scripting Tool (WLST) is one of the best utility which is based on Jython (The Java Version of Python Scripting) Option.

WLST not only allows to configure different resources in WebLogic Server/Domain as well as it allows a very easy way to monitor different resources available as part of WebLogic Server/Domain. We can monitor JMS/ DataSources/JDBC Parameters…etc using WLST.

Here we are going to see a very simple demonstration of using WLST to monitor Java Messaging Service (JMS) and the JMS Destination informations, which it holds, this part of monitoring includes both the Configuration Informations as well as the Runtime informations of the dataSource.

Step1). Create a WLST Script somewhere in your file system with some name like “Monitor_JMS.py”   inside “C:WLST_Jdbc”  contents will be something like following:

#############################################################################
#
# @author Copyright (c) 2010 - 2011 by Middleware Magic, All Rights Reserved.
#
#############################################################################

connect('weblogic','weblogic','t3://localhost:7001')
servers = domainRuntimeService.getServerRuntimes();
if (len(servers) > 0):
	for server in servers:
		jmsRuntime = server.getJMSRuntime();
		jmsServers = jmsRuntime.getJMSServers();
		for jmsServer in jmsServers:
			destinations = jmsServer.getDestinations();
			for destination in destinations:
				print '  BytesCurrentCount           ' ,  destination.getBytesCurrentCount()
				print '  BytesHighCount              ' ,  destination.getBytesHighCount()
				print '  BytesPendingCount           ' ,  destination.getBytesPendingCount()
				print '  BytesReceivedCount          ' ,  destination.getBytesReceivedCount()
				print '  BytesThresholdTime          ' ,  destination.getBytesThresholdTime()
				print '  ConsumersCurrentCount       ' ,  destination.getConsumersCurrentCount()
				print '  ConsumersHighCount          ' ,  destination.getConsumersHighCount()
				print '  ConsumersTotalCount         ' ,  destination.getConsumersTotalCount()
				print '  ConsumptionPausedState      ' ,  destination.getConsumptionPausedState()
				print '  '
				print '  DestinationInfo             ' ,  destination.getDestinationInfo()
				print '  '
				print '  DestinationType             ' ,  destination.getDestinationType()
				print '  InsertionPaused             ' ,  destination.isInsertionPaused()
				print '  InsertionPausedState        ' ,  destination.getInsertionPausedState()
				print '  MessagesCurrentCount        ' ,  destination.getMessagesCurrentCount()
				print '  MessagesDeletedCurrentCount ' ,  destination.getMessagesDeletedCurrentCount()
				print '  MessagesHighCount           ' ,  destination.getMessagesHighCount()
				print '  MessagesMovedCurrentCount   ' ,  destination.getMessagesMovedCurrentCount()
				print '  MessagesPendingCount        ' ,  destination.getMessagesPendingCount()
				print '  MessagesReceivedCount       ' ,  destination.getMessagesReceivedCount()
				print '  MessagesThresholdTime       ' ,  destination.getMessagesThresholdTime()
				print '  Parent                      ' ,  destination.getParent()
				print '  Paused                      ' ,  destination.isPaused()
				print '  ProductionPaused            ' ,  destination.isProductionPaused()
				print '  ProductionPausedState       ' ,  destination.getProductionPausedState()
				print '  State                       ' ,  destination.getState()
				print '  Type                        ' ,  destination.getType()

Step2). Start your WebLogic Server (Make sure that you have configured some JMS Server and JMS Modules in the Server to monitor). Or plesase refer to the following page to configure some JMS Resources like Queues/Topics:
http://middlewaremagic.com/weblogic/?page_id=1976

Step3). Open a command prompt and then run the “setWLSEnv.cmd” or “setWLSEnv.sh”  to set the CLASSPATH and PATH variables. Better you do echo %CLASSPATH%    or echo $CLASSPATH to see whether the CLASSPATH is set properly or not. If you see an Empty Classpath even after running the “setWLSEnv.sh” then please refer to the Note mentioned at Step3) in the Following post:   http://middlewaremagic.com/weblogic/?page_id=1492

Step4). Now run the WLST Script in the same command prompt like following:

java       weblogic.WLST        C:WLST_JdbcMonitor_JMS.py
You will see the following kind of results in the command prompt


  BytesCurrentCount            0
  BytesHighCount               0
  BytesPendingCount            0
  BytesReceivedCount           0
  BytesThresholdTime           0
  ConsumersCurrentCount        0
  ConsumersHighCount           0
  ConsumersTotalCount          0
  ConsumptionPausedState       Consumption-Enabled

  DestinationInfo              javax.management.openmbean.CompositeDataSupport(compositeType=javax.management.openmbean.CompositeType(name=DestinationInfo,items=((itemName=ApplicationName,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=ModuleName,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=Name,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=Queue,itemType=javax.management.openmbean.SimpleType(name=java.lang.Boolean)),(itemName=SerializedDestination,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=ServerName,itemType=javax.management.openmbean.SimpleType(name=java.lang.String)),(itemName=Topic,itemType=javax.management.openmbean.SimpleType(name=java.lang.Boolean)),(itemName=VersionNumber,itemType=javax.management.openmbean.SimpleType(name=java.lang.Integer)))),contents={ApplicationName=TestSystemModule, ModuleName=null, Name=TestSystemModule!Test_Q, Queue=true, SerializedDestination=rO0ABXNyACN3ZWJsb2dpYy5qbXMuY29tbW9uLkRlc3RpbmF0aW9uSW1wbFSmyJ1qZfv8DAAAeHB3gbRBABdUZXN0U3lzdGVtTW9kdWxlIVRlc3RfUQANVGVzdEpNU1NlcnZlcgAQVGVzdFN5c3RlbU1vZHVsZQEAA0FsbAICD/TBLPfeTooAAAAKAQALQWRtaW5TZXJ2ZXIALYQaAg/0wSz33k6KAAAADAEAC0FkbWluU2VydmVyAC2EGng=, ServerName=TestJMSServer, Topic=false, VersionNumber=1})

  DestinationType              Queue
  InsertionPaused              0
  InsertionPausedState         Insertion-Enabled
  MessagesCurrentCount         0
  MessagesDeletedCurrentCount  0
  MessagesHighCount            0
  MessagesMovedCurrentCount    0
  MessagesPendingCount         0
  MessagesReceivedCount        0
  MessagesThresholdTime        0
  Name                         TestSystemModule!Test_Q
  Parent                       [MBeanServerInvocationHandler]com.bea:ServerRuntime=AdminServer,Name=TestJMSServer,Location=AdminServer,Type=JMSServerRuntime
  Paused                       0
  ProductionPaused             0
  ProductionPausedState        Production-Enabled
  State                        advertised_in_cluster_jndi
  Type                         JMSDestinationRuntime
JMS_Monitoring_Using_WLST

JMS_Monitoring_Using_WLST

.

.

Thanks

Jay SenSharma


WebLogic DataSource Monitoring Using WLST

Hi,

Jay SenSharma

Jay SenSharma

Many times we want to monitor the different modules available as part of our WebLogic Domain. WebLogic Scripting Tool (WLST) is one of the best utility which is based on Jython (The Java Version of Python Scripting) Option.

WLST not only allows to configure different resources in WebLogic Server/Domain as well as it allows a very easy way to monitor different resources available as part of WebLogic Server/Domain. We can monitor JMS/ DataSources/JDBC Parameters…etc using WLST.

Here we are going to see a very simple demonstration of using WLST to monitor DataSources and the JDBC Properties which it holds, this part of monitoring includes both the Configuration Informations as well as the Runtime informations of the dataSource.

Step1). Create a WLST Script somewhere in your file system with some name like “Monitor_Jdbc.py”   inside “C:WLST_Jdbc”  contents will be something like following:

#############################################################################
#
# @author Copyright (c) 2010 - 2011 by Middleware Magic, All Rights Reserved.
#
#############################################################################

connect('weblogic','weblogic','t3://localhost:7001')
allServers=domainRuntimeService.getServerRuntimes();
if (len(allServers) > 0):
  for tempServer in allServers:
    jdbcServiceRT = tempServer.getJDBCServiceRuntime();
    dataSources = jdbcServiceRT.getJDBCDataSourceRuntimeMBeans();
    if (len(dataSources) > 0):
		for dataSource in dataSources:
			print 'ActiveConnectionsAverageCount      '  ,  dataSource.getActiveConnectionsAverageCount()
			print 'ActiveConnectionsCurrentCount      '  ,  dataSource.getActiveConnectionsCurrentCount()
			print 'ActiveConnectionsHighCount         '  ,  dataSource.getActiveConnectionsHighCount()
			print 'ConnectionDelayTime                '  ,  dataSource.getConnectionDelayTime()
			print 'ConnectionsTotalCount              '  ,  dataSource.getConnectionsTotalCount()
			print 'CurrCapacity                       '  ,  dataSource.getCurrCapacity()
			print 'CurrCapacityHighCount              '  ,  dataSource.getCurrCapacityHighCount()
			print 'DeploymentState                    '  ,  dataSource.getDeploymentState()
			print 'FailedReserveRequestCount          '  ,  dataSource.getFailedReserveRequestCount()
			print 'FailuresToReconnectCount           '  ,  dataSource.getFailuresToReconnectCount()
			print 'HighestNumAvailable                '  ,  dataSource.getHighestNumAvailable()
			print 'HighestNumUnavailable              '  ,  dataSource.getHighestNumUnavailable()
			print 'LeakedConnectionCount              '  ,  dataSource.getLeakedConnectionCount()
			print 'ModuleId                           '  ,  dataSource.getModuleId()
			print 'Name                               '  ,  dataSource.getName()
			print 'NumAvailable                       '  ,  dataSource.getNumAvailable()
			print 'NumUnavailable                     '  ,  dataSource.getNumUnavailable()
			print 'Parent                             '  ,  dataSource.getParent()
			print 'PrepStmtCacheAccessCount           '  ,  dataSource.getPrepStmtCacheAccessCount()
			print 'PrepStmtCacheAddCount              '  ,  dataSource.getPrepStmtCacheAddCount()
			print 'PrepStmtCacheCurrentSize           '  ,  dataSource.getPrepStmtCacheCurrentSize()
			print 'PrepStmtCacheDeleteCount           '  ,  dataSource.getPrepStmtCacheDeleteCount()
			print 'PrepStmtCacheHitCount              '  ,  dataSource.getPrepStmtCacheHitCount()
			print 'PrepStmtCacheMissCount             '  ,  dataSource.getPrepStmtCacheMissCount()
			print 'Properties                         '  ,  dataSource.getProperties()
			print 'ReserveRequestCount                '  ,  dataSource.getReserveRequestCount()
			print 'State                              '  ,  dataSource.getState()
			print 'Type                               '  ,  dataSource.getType()
			print 'VersionJDBCDriver                  '  ,  dataSource.getVersionJDBCDriver()
			print 'WaitingForConnectionCurrentCount   '  ,  dataSource.getWaitingForConnectionCurrentCount()
			print 'WaitingForConnectionFailureTotal   '  ,  dataSource.getWaitingForConnectionFailureTotal()
			print 'WaitingForConnectionHighCount      '  ,  dataSource.getWaitingForConnectionHighCount()
			print 'WaitingForConnectionSuccessTotal   '  ,  dataSource.getWaitingForConnectionSuccessTotal()
			print 'WaitingForConnectionTotal          '  ,  dataSource.getWaitingForConnectionTotal()
			print 'WaitSecondsHighCount               '  ,  dataSource.getWaitSecondsHighCount()

Step2). Start your WebLogic Server and Databases (Make sure that you have configured some dataSources in the Server to monitor).

Step3). Open a command prompt and then run the “setWLSEnv.cmd” or “setWLSEnv.sh”  to set the CLASSPATH and PATH variables. Better you do echo %CLASSPATH%    or echo $CLASSPATH to see whether the CLASSPATH is set properly or not. If you see an Empty Classpath even after running the “setWLSEnv.sh” then please refer to the Note mentioned at Step3) in the Following post:   http://middlewaremagic.com/weblogic/?page_id=1492

Step4). Now run the WLST Script in the same command prompt like following:

java       weblogic.WLST        C:WLST_JdbcMonitor_Jdbc.py
You will see the following kind of results in the command prompt

ActiveConnectionsAverageCount       0
ActiveConnectionsCurrentCount       0
ActiveConnectionsHighCount          0
ConnectionDelayTime                 25
ConnectionsTotalCount               10
CurrCapacity                        10
CurrCapacityHighCount               1
DeploymentState                     2
FailedReserveRequestCount           0
FailuresToReconnectCount            0
HighestNumAvailable                 10
HighestNumUnavailable               0
LeakedConnectionCount               0
ModuleId                            TestDS
Name                                TestDS
NumAvailable                        10
NumUnavailable                      0
Parent                              [MBeanServerInvocationHandler]com.bea:Name=AdminServer,Location=AdminServer,Type=ServerRuntime
PrepStmtCacheAccessCount            0
PrepStmtCacheAddCount               0
PrepStmtCacheCurrentSize            0
PrepStmtCacheDeleteCount            0
PrepStmtCacheHitCount               0
PrepStmtCacheMissCount              0
Properties                          {databaseName=jdbc:pointbase:server://localhost:9092/demo, user=PBPUBLIC}
ReserveRequestCount                 0
State                               Running
Type                                JDBCDataSourceRuntime
VersionJDBCDriver                   com.pointbase.jdbc.jdbcUniversalDriver
WaitingForConnectionCurrentCount    0
WaitingForConnectionFailureTotal    0
WaitingForConnectionHighCount       0
WaitingForConnectionSuccessTotal    0
WaitingForConnectionTotal           0
WaitSecondsHighCount                0
Jdbc_Monitoring_Using_WLST

Jdbc_Monitoring_Using_WLST

.

.

Thanks

Jay SenSharma


Copyright © 2010-2012 Middleware Magic. All rights reserved. |