From WLS10.3 onwards enabling and disabling feature of Lock & Edit button is added.
Login to AdminConsole—> Click on (Preference) link —-> User preferences (Tab) —->Bottom off the page you will see a check box: Automatically Acquire Lock and Activate Changes
Try to check and Uncheck that Field in production and development Mode…and then check if you see any differences in both scenarios?
I’ve one question , but not sure whether i can ask here or not..
A very important part of WebLogic Administration is to do with resolving and troubleshooting issues most notably in Production.
Our value is determined by how well we diagnose the problems, bring the services up, determine the root cause of the issue and prevent it from happening in future.
The following are typical reports from users for issues
* The application does not respond and browser just clocks
* Users receiving a page that says Internal Server intermittently
* Users receiving pages that have “404” – Page not found intermittently
* Users getting Page cannot be displayed errors intermittently
* The application is being very slow than usual
What are basic/common approaches to these types of problems…?
Hi Arun,
There is no hard runles to investigate these kind of issues because every time for these kind of issues we find different root causes…. It can be solved by analyzing the Logs/out/debugging only. How to start debugging these issue …is simply “Look at the server logs…if you dont find much informations then enable debug flags to get more informations….etc”
* The application does not respond and browser just clocks …Take 5-6 Thread Dumps in the interval of 10-12 seconds each… and analyze them.
* Users receiving a page that says Internal Server intermittently Internal Server Error or Error Code 500 in the browser means you need to look into the server Logs to find out the root cause of failure
* Users receiving pages that have “404″ – Page not found intermittently Check the “access.log” request is reaching to server or not for the correct resource…Check the URL is correct or not? Check the Server Logs for any kind of Application related WARNNING or Error Message. You can even check the application logs (if you have any).
* Users getting Page cannot be displayed errors intermittently. Check the “access.log” request is reaching to server or not for the correct resource…Check the URL is correct or not? Check the Server Logs for any kind of Application related WARNNING or Error Message. You can even check the application logs (if you have any).
* The application is being very slow than usual. Take 5-6 Thread Dumps in the interval of 10-12 seconds each… and analyze them.
Hi Jay
I will first say Thanks to you and faisal for sharing such a good info about weblogic . This site will helping to everyone who intrested in weblogic administration.
I need your help to understand Thread Dump(TD).
As per your post there are many ways we can take thread dumps, But how we can understand it.
for eg I am pasting here some part of TD.
requesting you to please provide your inputs for same.
=======================================================
Sorry for delay in response. I donot see anything Unusual in the Above Thread Dump. Threads are locking different objects and releasing them as well…which is normal. Apart from that regarding tid=30097
“ExecuteThread: ‘1′ for queue: ‘weblogic.socket.Muxer’” id=21 idx=0×6c tid=30097 (active)
I can see that from weblogic.socket.Muxer Queue the Execute thread 1 is Active …which is normal. We dont expect that all the Muxer threads will be active at the same time….Some of them must be in the waiting state…
What SocketMuxers Do ?
The socket Muxer manages the server’s existing socket connections.
It first determines which sockets have incoming requests waiting to be processed. It then reads enough data to determine the protocol and dispatches the socket to an appropriate runtime layer based on the protocol.In the runtime layer, the socket muxer threads determine which execute thread queue to be used and delegates the request accordingly.
Weblogic has two versions of the socket muxer, one is the Java version and the other uses a native library which makes better use of operating system calls. Native Version — The Enable Native IO checkbox on the server’s configuration settings tells the server which version to use. This is ON by default for most platforms.
Analyzing One thread dump i not very useful…We should always collect at least 5-6 Thread Dumps taken in the interval of 9-10 Seconds each.
Ravi Also can u please specify which kind of issue you are facing and Like of Stuck Thread or Slow Response….Because from the Thread Dump i cannot see any such issue.
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
Hopefully someone can shed some light on my issue, I can’t believe I am the first to see this.
I have a working set of build scripts for creation and configuration of weblogic domains. Previously I have been able to put default passwords, DB passwords etc into a set of property files for the scripts to configure against.
New security regulations mean that I now need to either prompt the user building the domains to input a password (this cannot be echoed in anyway), encrypt it pre-configuration or have it as a manual task post installation. (The latter being the least desirable option)
I downloaded jython and got hold of getpass as trawling the internet this looked like a good viable option. Unfortunately when getpass is invoked I get
“Warning: Problem with getpass. Passwords may be echoed”)
….and sure enough they are.
Has anyone had to do anything like this before? Essentially I am trying to simply mask raw_input.
I fail to believe I am the first that’s had this issue. I am using WLS11Gr1 on jrockit 1.6.0_14 (Windows) and Sun 1.5.0_23 (Solaris)
Any help or suggestions would be greatly appreciated.
import os
import sys
import threading
from java.io import *
class MaskingPassword(threading.Thread):
# This thread will take over control and put a * for every character you type.
# If you try to type characters too fast for a test, you might see some in clear text, but that's not the human speed to type a password.
def __init__ (self):
threading.Thread.__init__(self)
self.stop_event = threading.Event()
def stop(self):
self.stop_event.set()
def run(self):
Thread.currentThread().setPriority(Thread.MAX_PRIORITY)
while not self.stop_event.isSet():
sys.stdout.write("10" + "*")
Thread.currentThread().sleep(1)
userId = raw_input('Enter username:')
MaskingThread = MaskingPassword()
MaskingThread.start()
passWd = raw_input("Enter password: ")
MaskingThread.stop()
I am setting Oracle Fusion Middleware env on a RHEL 5.3.
I have done with the installation of Oracle Weblogic Server and try to setup the identity management suite. I have the following queries.
1) I am able to install the Oracle Weblogic with root. But for others like IDM Suite, Oracle webtier utilities the installation failed saying “Installation with root is not allowed.” I am not sure why
2) After I have created the reqd users and groups I am able to start the IDM installation but I got the following pre-req package checks failed.
Checking for compat-libstdc++-33-3.2.3-x86_64; Not found. Failed <<<<
Checking for compat-libstdc++-33-3.2.3-i386; Not found. Failed <<<<
Checking for elfutils-libelf-devel-0.125; Not found. Failed <<<<
Checking for gcc-4.1.1; Not found. Failed <<<<
Checking for gcc-c++-4.1.1; Not found. Failed <<<<
Checking for glibc-devel-2.5-x86_64; Not found. Failed <<<<
Checking for glibc-devel-2.5-12-i386; Not found. Failed <<<<
Checking for libaio-devel-0.3.106; Not found. Failed <<<<
Checking for libstdc++-devel-4.1.1; Not found. Failed <<<<
Checking for sysstat-7.0.0; Not found. Failed <<<<
Check complete. The overall result of this check is: Failed <<<<
How I can get and Install the libs for which the check has failed above?
initial you have create the directory in your configuration log file path
#cd /oracle
#mkdir adf_log
Default path for Managed server log:
Log file name: logs/MS1.log
The managed server default log path is ” /oracle/Middleware/user_projects/domain/domainname/servers/MS1/logs/ ”
Changed into our configured path:
Log file name: /oracle/adf_log/
you have configure the samba service into the newly created adf_log directory to provide read permisson. So all the Developer can see the log file in the Windows environment.
#cd /etc/samba/
#vi smb.conf
security = share
[adf_log]
path = /oracle/adf_log
guest ok = yes
The “Limit number of retained files” default option is unmarked.
But you have to limit the log file to store the hard disk. Because in our dev , test environment it occupy more than 30 GB space. So to avoid these storage in the Weblogic Application Server side.
Follow the below steps:-
Limit number of retained files –> mark
Files to retain: –> 50 ( it take 50 backup of MS1 log )
you have to create the “backup” directory inside the adf_log path.
#cd /oracle/adf_log
#mkdir backup
So you have to limit the Managed server log file upto 50 * 5 MB = 250 MB.
The main advantage is you have to see the latest file available in the “/oracle/adf_log/ “path
The previous log file are available in the ” /oracle/adf_log/backup/ “path. It store the backup of 50 managed server MS1 logs.
The same steps you have to configure for every managed server. To avoid the memory area in the Weblogic Application Server.
First I wanted to thank you for putting together such a fantastic resource! This blog is immensely useful!
—
Issue: WLST Help
OS: RedHat Enterprise Linux 5 – 32bit
WLS: 10.3.3
JDK: jrockit_160_05_R27.6.2-20 , Xms1024 Xmx1024
I’m looking to create some WLST scripts which stop a set of servers, then start them again. These particular servers tend to come up in Admin mode because of connectivity issues with remote databases. I’m not having any luck getting these to resume from Admin mode. Does anyone have any suggestions?
<RuntimeException thrown by rmi server: weblogic.management.intern
al.RemoteMBeanServerImpl.getAttribute(Ljavax.management.ObjectName;Ljava.lang.String;)
java.lang.IllegalArgumentException: Object name for an MBean can not be null.
java.lang.IllegalArgumentException: Object name for an MBean can not be null
Can you please help me to give a fix for this. Thanks.
Hi Rajendra,
There is no specific reason for this WARNING. Investigation on this will require more detailed and in depth analysis of all the logs and the configuration …as well as what happed at the time when the WARNNING was occurred like was there any recent changes … We would recommend you to please Open A Oracle WebLogic Support ticket as well as additionally post the same query on Oracle Forums: http://forums.oracle.com/forums/forum.jspa?forumID=570 To get more quick and accurate results.
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
I am new to Middleware Magic and not sure how to create a new post as I dont even seem to have access to search which was available earlier. I wanted to find out about Java process eating my CPU resources (Weblogic server with JRockit on Linux). What are some of the basic troubleshooting tips to find out whats causing the java process to tale almost 100 % CPU?
Apologies if this is not the right way for posting any new issues or queries.
Can someone help me in understanding the concept “Deployment Order” ?
To my knowledge i can brief certain things like –
1. The default deployment order is 100
2. Usually the order the way deployments works –
(a) JDBC
(b) JMS
(c) J2EE libraries , and optional packages if any
(d) Web applications and actual applications
(e) Startup classes
In my project i observed that one of the my application deployment had the order “100” , and some other with “161”
I just want to understand like –
1) Deployment order value 100 means ?
2) And what’s the value 161 says then ?
3) How to differentiate both?
4) Or purpose/use of this deployment order.
I have the following error in the logs. Could you please suggest. What is the Socket IP and port that it is pointing to, i am not aware of that IP:port, whats that exactly?
#### <> <IOException occurred on socket: Socket[addr=/10.10.175.14,port=2096,localport=7017]
java.net.SocketException: Connection refused.
java.net.SocketException: Connection refused
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at weblogic.socket.SocketMuxer.readReadySocketOnce(SocketMuxer.java:842)
at weblogic.socket.SocketMuxer.readReadySocket(SocketMuxer.java:818)
at weblogic.socket.DevPollSocketMuxer.processSockets(DevPollSocketMuxer.java:120)
at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
Hi Manishy,
This is true for every transaction, That the application level liek ejbs/jms/mdb or Jdbc level transactions timeout value should be greater than or equal to the global WebLogic Server transaction timeout. Configuration settings for JTA (transactions) are applicable at the domain level. This means that configuration attribute settings apply to all servers within a domain.
I was monitoring the Heap Size and i observed that the Heap is growing to 100% and after Full GC it is settling down to 35-40% , could you please explain this behavior and i welcome your suggestions if you have any. Is it normal that the Heap settling down to 40% of its capacity? Here is an output of jstat that i took for every 10 minutes.
“I was monitoring the Heap Size and i observed that the Heap is growing to 100% and after Full GC it is settling down to 35-40% , could you please explain this behavior”
This is the way for the JVM to optimize for throughput, i.e., giving the Java threads as much processing time as possible. In this case minor collections occur frequenlty, until the memory reaches a certain limit, usually about 95% it starts planning for a full collection. A nice illustration of this behavior is depicted here: http://middlewaremagic.com/weblogic/wp-content/uploads/2011/07/clientromemoryoverview.png
“Is it normal that the Heap settling down to 40% of its capacity?”
This depends on the number of live objects in your heap, the garbage collector only collects ‘dead’ object (objects that are not referenced any longer.
Hi Kiran,
Usually WebLogic Server deploys server-level resources (first JDBC and then JMS) before deploying application modules. Modules are then deployed in order starting with connector modules, followed by EJB modules, and finally Web Applications.
The actual deployment order of modules is determined by their Load Order attribute. By default, new applications and modules are configured with a Load Order value of 100. Modules with a lower Load Order value are deployed before those with a higher value during startup. Modules with the same Load Order value are deployed in alphabetical order using the deployment name.
You can change the deployment order for modules from the admin console. The actual deployment order of modules is determined by their Deployment Order attribute.
NOTE: But in many times it is required that we need to set the deployment order for different modules on our own … if we see any issue.
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
I’m investigating an issue that occurred in our weblogic in production. The weblogic console hangs and the cpu utilization is high. Can you help me analyze what possible cause of this? Here’s thread dump:
“Main Thread” prio=5 id=0x80 pid=11252 waiting
— Waiting for notification on: weblogic/t3/srvr/T3Srvr@0x20eef878[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:1207)@0xb3805e31
^– Lock released while waiting: weblogic/t3/srvr/T3Srvr@0x20eef878[fat lock]
at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:389)@0xb73bac31
at weblogic/Server.main(Server.java:32)@0xb73ba584
— End of stack trace
“Finalizer” prio=8 id=0x700 pid=11265 waiting, daemon
— Waiting for notification on: java/lang/ref/ReferenceQueue$Lock@0x20088650[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at jrockit/vm/Locks.wait(Optimized Method)@0xb223b03d
at java/lang/ref/ReferenceQueue.remove(ReferenceQueue.java:111)@0xb7399611
^– Lock released while waiting: java/lang/ref/ReferenceQueue$Lock@0x20088650[fat lock]
at java/lang/ref/ReferenceQueue.remove(ReferenceQueue.java:127)@0xb739950c
at java/lang/ref/Finalizer$FinalizerThread.run(Unknown Source)@0xb7399464
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“Reference Handler” prio=10 id=0x780 pid=11266 waiting for pending references, daemon
at java/lang/ref/Reference.getPending(Native Method)@0xb739a3e0
at java/lang/ref/Reference.access$000(Unknown Source)@0xb739a455
at java/lang/ref/Reference$ReferenceHandler.run(Unknown Source)@0xb739a360
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“Thread-1” prio=5 id=0x880 pid=11272 waiting, daemon
— Waiting for notification on: java/util/TaskQueue@0x21482530[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at java/util/TimerThread.mainLoop(Timer.java:403)@0xb68b6ff0
^– Lock released while waiting: java/util/TaskQueue@0x21482530[fat lock]
at java/util/TimerThread.run(Timer.java:382)@0xb68b6f12
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x900 pid=11273 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c3feb0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c3feb0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘1’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x980 pid=11274 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c405e0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c405e0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘2’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xa00 pid=11275 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c40b28[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c40b28[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘3’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xa80 pid=11276 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c41070[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c41070[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘4’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xb00 pid=11277 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c415b8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c415b8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘5’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xb80 pid=11278 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c41b00[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c41b00[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘6’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xc00 pid=11279 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c42048[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c42048[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘7’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xc80 pid=11280 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c42590[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c42590[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘8’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xd00 pid=11281 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c42ad8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c42ad8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘9’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xd80 pid=11282 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c43020[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c43020[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’10’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xe00 pid=11283 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c43568[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c43568[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’11’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xe80 pid=11284 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c43ab0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c43ab0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’12’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xf00 pid=11285 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c43ff8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c43ff8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’13’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xf80 pid=11286 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c44540[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c44540[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’14’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1000 pid=11287 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c44a88[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c44a88[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’15’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1080 pid=11288 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c44fd0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c44fd0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’16’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1100 pid=11289 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c45518[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c45518[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’17’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1180 pid=11290 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c45af0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c45af0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’18’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1200 pid=11291 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c46038[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c46038[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’19’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1280 pid=11292 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c46580[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c46580[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’20’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1300 pid=11293 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c46ac8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c46ac8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’21’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1380 pid=11294 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c47010[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c47010[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’22’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1400 pid=11295 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c47558[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c47558[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’23’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1480 pid=11296 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c47aa0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c47aa0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’24’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1500 pid=11297 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c47fa0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c47fa0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘weblogic.kernel.System'” prio=5 id=0x1580 pid=11298 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c49208[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c49208[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘1’ for queue: ‘weblogic.kernel.System'” prio=5 id=0x1600 pid=11299 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c49740[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c49740[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘2’ for queue: ‘weblogic.kernel.System'” prio=5 id=0x1680 pid=11300 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c49c78[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c49c78[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘3’ for queue: ‘weblogic.kernel.System'” prio=5 id=0x1700 pid=11301 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c4a1b0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c4a1b0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘4’ for queue: ‘weblogic.kernel.System'” prio=5 id=0x1780 pid=11302 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c4a6e8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c4a6e8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“weblogic.time.TimeEventGenerator” prio=9 id=0x1800 pid=11303 waiting, daemon
— Waiting for notification on: weblogic/time/common/internal/TimeTable@0x23c4b090[fat lock]
at jrockit/vm/Threads.waitForSignalWithTimeout(Native Method)@0xb738e190
at jrockit/vm/Locks.wait(Optimized Method)@0xb223b454
at weblogic/time/common/internal/TimeTable.snooze(Optimized Method)@0xb2290e84
^– Lock released while waiting: weblogic/time/common/internal/TimeTable@0x23c4b090[fat lock]
at weblogic/time/common/internal/TimeEventGenerator.run(TimeEventGenerator.java:118)@0xb5cfb184
at java/lang/Thread.run(Unknown Source)@0xb690a62f
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“weblogic.security.SpinnerRandomSource” prio=5 id=0x1880 pid=11304 waiting, daemon
— Waiting for notification on: java/lang/Object@0x2329ffc8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/security/SpinnerRandomBitsSource.run(SpinnerRandomBitsSource.java:60)@0xb5d01533
^– Lock released while waiting: java/lang/Object@0x2329ffc8[fat lock]
at java/lang/Thread.run(Unknown Source)@0xb690a62f
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘weblogic.admin.HTTP'” prio=5 id=0x1900 pid=11305 active, daemon
at java/util/HashMap.get(Optimized Method)@0xb69471be
at weblogic/management/console/info/AttributeRegistry.getAttribute(AttributeRegistry.java:85)@0xb30d5fe6
at weblogic/management/console/info/Attribute$Factory.getInstance(Attribute.java:308)@0xb30d5a8a
at weblogic/management/console/info/CompositeAttribute.(CompositeAttribute.java:58)@0xb30dc9dd
at weblogic/management/console/info/Attribute$Factory.getInstance(Attribute.java:314)@0xb30d5f36
at weblogic/management/console/tags/form/ControlTag.setAttribute(ControlTag.java:77)@0xb30d58cc
at weblogic/management/console/webapp/_domain/__machine._jspService(__machine.java:520)@0xb26fa684
at weblogic/servlet/jsp/JspBase.service(JspBase.java:33)@0xb384257b
at weblogic/servlet/internal/ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1072)@0xb3842525
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:465)@0xb3841b24
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:348)@0xb3841868
at weblogic/servlet/internal/RequestDispatcherImpl.forward(RequestDispatcherImpl.java:322)@0xb3840c04
at weblogic/servlet/jsp/PageContextImpl.forward(PageContextImpl.java:150)@0xb386118a
at weblogic/management/console/actions/ForwardAction.perform(ForwardAction.java:35)@0xb3861126
at weblogic/management/console/actions/internal/ActionServlet.doAction(ActionServlet.java:182)@0xb385b10b
at weblogic/management/console/actions/internal/ActionServlet.doGet(ActionServlet.java:92)@0xb385ac10
at javax/servlet/http/HttpServlet.service(HttpServlet.java:740)@0xb385ab25
at javax/servlet/http/HttpServlet.service(HttpServlet.java:853)@0xb385a89e
at weblogic/servlet/internal/ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1072)@0xb3842525
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:465)@0xb3841b24
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:348)@0xb3841868
at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6981)@0xb385a768
at weblogic/security/acl/internal/AuthenticatedSubject.doAs(Optimized Method)@0xb2f1ef3b
at weblogic/security/service/SecurityManager.runAs(Optimized Method)@0xb3882c27
at weblogic/servlet/internal/WebAppServletContext.invokeServlet(WebAppServletContext.java:3892)@0xb3819d24
at weblogic/servlet/internal/ServletRequestImpl.execute(ServletRequestImpl.java:2766)@0xb381883e
at weblogic/kernel/ExecuteThread.execute(Optimized Method)@0xb3895a70
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:183)@0xb5cfa8ce
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘1’ for queue: ‘weblogic.admin.HTTP'” prio=5 id=0x1980 pid=11306 active, daemon
at java/util/HashMap.get(Optimized Method)@0xb69471be
at weblogic/management/console/info/AttributeRegistry.getAttribute(Optimized Method)@0xb2284521
at weblogic/management/console/info/Attribute$Factory.getInstance(Optimized Method)@0xb2284561
at weblogic/management/console/info/CompositeAttribute.(CompositeAttribute.java:58)@0xb30dc9dd
at weblogic/management/console/info/Attribute$Factory.getInstance(Optimized Method)@0xb2284b08
at weblogic/management/console/tags/form/ControlTag.setAttribute(ControlTag.java:77)@0xb30d58cc
at weblogic/management/console/webapp/_domain/__machine._jspService(__machine.java:520)@0xb26fa684
at weblogic/servlet/jsp/JspBase.service(JspBase.java:33)@0xb384257b
at weblogic/servlet/internal/ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1072)@0xb3842525
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:465)@0xb3841b24
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:348)@0xb3841868
at weblogic/servlet/internal/RequestDispatcherImpl.forward(RequestDispatcherImpl.java:322)@0xb3840c04
at weblogic/servlet/jsp/PageContextImpl.forward(PageContextImpl.java:150)@0xb386118a
at weblogic/management/console/actions/ForwardAction.perform(ForwardAction.java:35)@0xb3861126
at weblogic/management/console/actions/internal/ActionServlet.doAction(ActionServlet.java:182)@0xb385b10b
at weblogic/management/console/actions/internal/ActionServlet.doGet(ActionServlet.java:92)@0xb385ac10
at javax/servlet/http/HttpServlet.service(HttpServlet.java:740)@0xb385ab25
at javax/servlet/http/HttpServlet.service(HttpServlet.java:853)@0xb385a89e
at weblogic/servlet/internal/ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1072)@0xb3842525
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:465)@0xb3841b24
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:348)@0xb3841868
at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6981)@0xb385a768
at weblogic/security/acl/internal/AuthenticatedSubject.doAs(Optimized Method)@0xb2f1ef3b
at weblogic/security/service/SecurityManager.runAs(Optimized Method)@0xb3882c27
at weblogic/servlet/internal/WebAppServletContext.invokeServlet(WebAppServletContext.java:3892)@0xb3819d24
at weblogic/servlet/internal/ServletRequestImpl.execute(ServletRequestImpl.java:2766)@0xb381883e
at weblogic/kernel/ExecuteThread.execute(Optimized Method)@0xb3895a70
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:183)@0xb5cfa8ce
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘weblogic.admin.RMI'” prio=5 id=0x1a00 pid=11307 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23ca8fe0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23ca8fe0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘1’ for queue: ‘weblogic.admin.RMI'” prio=5 id=0x1a80 pid=11308 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23ca9528[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23ca9528[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘2’ for queue: ‘weblogic.admin.RMI'” prio=5 id=0x1b00 pid=11309 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23ca9a50[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23ca9a50[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“VDE Transaction Processor Thread” prio=2 id=0x1b80 pid=11310 waiting
— Waiting for notification on: com/octetstring/vde/backend/standard/TransactionProcessor@0x23265b00[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at com/octetstring/vde/backend/standard/TransactionProcessor.waitChange(TransactionProcessor.java:365)@0xb5d16455
^– Lock released while waiting: com/octetstring/vde/backend/standard/TransactionProcessor@0x23265b00[fat lock]
at com/octetstring/vde/backend/standard/TransactionProcessor.run(TransactionProcessor.java:212)@0xb5d155bd
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“VDE Replication Thread” prio=5 id=0x1c00 pid=11311 waiting
— Waiting for notification on: com/octetstring/vde/replication/Replication@0x23ce4008[fat lock]
at jrockit/vm/Threads.waitForSignalWithTimeout(Native Method)@0xb738e190
at jrockit/vm/Locks.wait(Optimized Method)@0xb223b454
at com/octetstring/vde/replication/Replication.wait30sec(Replication.java:395)@0xb5d1cabe
^– Lock released while waiting: com/octetstring/vde/replication/Replication@0x23ce4008[fat lock]
at com/octetstring/vde/replication/Replication.run(Replication.java:365)@0xb5d1c97d
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“Thread-5” prio=5 id=0x1c80 pid=11313 waiting
— Waiting for notification on: java/util/TaskQueue@0x25a060e0[fat lock]
at jrockit/vm/Threads.waitForSignalWithTimeout(Native Method)@0xb738e190
at jrockit/vm/Locks.wait(Optimized Method)@0xb223b454
at java/util/TimerThread.mainLoop(Timer.java:429)@0xb68b7129
^– Lock released while waiting: java/util/TaskQueue@0x25a060e0[fat lock]
at java/util/TimerThread.run(Timer.java:382)@0xb68b6f12
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“[JRockit] GCEventThread” prio=5 id=0x1d00 pid=11314 active, native_waiting, daemon
at jrockit/management/jvm/GarbageCollectorImpl.waitForEvent(Native Method)@0xb4c19840
at jrockit/management/jvm/GarbageCollectorImpl.access$000(Unknown Source)@0xb4c198b5
at jrockit/management/jvm/GarbageCollectorImpl$GCEventThread.getNextEvent(Unknown Source)@0xb4c1978e
at jrockit/management/jvm/AsyncEventProvider$EventThread.run(Unknown Source)@0xb4c196d2
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“weblogic.health.CoreHealthMonitor” prio=5 id=0x1d80 pid=11315 waiting, daemon
— Waiting for notification on: java/lang/Object@0x2642dd10[fat lock]
at jrockit/vm/Threads.waitForSignalWithTimeout(Native Method)@0xb738e190
at jrockit/vm/Locks.wait(Optimized Method)@0xb223b454
at java/lang/Thread.sleep(Unknown Source)@0xb690a3d5
^– Lock released while waiting: java/lang/Object@0x2642dd10[fat lock]
at weblogic/t3/srvr/CoreHealthMonitorThread.run(CoreHealthMonitorThread.java:163)@0xb4c49b60
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘JMS.TimerTreePool'” prio=5 id=0x1e00 pid=11316 waiting, daemon
— Waiting for notification on: weblogic/jms/backend/BETimerTree@0x21eb7140[fat lock]
at jrockit/vm/Threads.waitForSignalWithTimeout(Native Method)@0xb738e190
at weblogic/jms/backend/BETimerTree.execute(Optimized Method)@0xb3894e8d
^– Lock released while waiting: weblogic/jms/backend/BETimerTree@0x21eb7140[fat lock]
at weblogic/kernel/ExecuteThread.execute(Optimized Method)@0xb3895a70
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:183)@0xb5cfa8ce
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“Thread-6” prio=5 id=0x1e80 pid=11317 waiting, daemon
— Waiting for notification on: java/util/TaskQueue@0x21f76f70[fat lock]
at jrockit/vm/Threads.waitForSignalWithTimeout(Native Method)@0xb738e190
at jrockit/vm/Locks.wait(Optimized Method)@0xb223b454
at java/util/TimerThread.mainLoop(Timer.java:429)@0xb68b7129
^– Lock released while waiting: java/util/TaskQueue@0x21f76f70[fat lock]
at java/util/TimerThread.run(Timer.java:382)@0xb68b6f12
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ListenThread.Default” prio=5 id=0x1f00 pid=11318 active
at jrockit/net/SocketNativeIO.accept(Native Method)@0xb37ebb70
at jrockit/net/SocketNativeIO.accept(Unknown Source)@0xb37ebc3b
at java/net/AbstractSocketImpl.accept(Unknown Source)@0xb37eba6b
^– Holding lock: java/net/PlainSocketImpl@0x23288a70[thin lock]
at java/net/ServerSocket.implAccept(ServerSocket.java:448)@0xb37eb973
at java/net/ServerSocket.accept(ServerSocket.java:419)@0xb37eb86d
at weblogic/socket/WeblogicServerSocket.accept(WeblogicServerSocket.java:26)@0xb38031fa
at weblogic/t3/srvr/ListenThread.accept(ListenThread.java:735)@0xb38031d7
at weblogic/t3/srvr/ListenThread.run(ListenThread.java:301)@0xb37de904
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“SSLListenThread.Default” prio=5 id=0x1f80 pid=11319 active
at jrockit/net/SocketNativeIO.accept(Native Method)@0xb37ebb70
at jrockit/net/SocketNativeIO.accept(Unknown Source)@0xb37ebc3b
at java/net/AbstractSocketImpl.accept(Unknown Source)@0xb37eba6b
^– Holding lock: java/net/PlainSocketImpl@0x232aad80[thin lock]
at java/net/ServerSocket.implAccept(ServerSocket.java:448)@0xb37eb973
at javax/net/ssl/impl/SSLServerSocketImpl.accept(Unknown Source)@0xb38032b4
at weblogic/t3/srvr/ListenThread.accept(ListenThread.java:735)@0xb38031d7
at weblogic/t3/srvr/ListenThread.run(ListenThread.java:301)@0xb37de904
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘weblogic.socket.Muxer'” prio=5 id=0x2000 pid=11320 active, daemon
at weblogic/socket/PosixSocketMuxer.poll(Native Method)@0xb3810b40
at weblogic/socket/PosixSocketMuxer.processSockets(PosixSocketMuxer.java:100)@0xb38003fc
^– Holding lock: java/lang/String@0x23282900[fat lock]
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:32)@0xb380033a
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:224)@0xb5dbfa10
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:183)@0xb5cfa8ce
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” prio=5 id=0x2080 pid=11321 blocked, daemon
— Blocked trying to get lock: java/lang/String@0x23282900[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.lockFat(Optimized Method)@0xb312a2e9
at jrockit/vm/Locks.monitorEnterSecondStage(Optimized Method)@0xb3126e44
at jrockit/vm/Locks.monitorEnter(Native Method)@0xb738b694
at weblogic/socket/PosixSocketMuxer.processSockets(PosixSocketMuxer.java:91)@0xb380039c
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:32)@0xb380033a
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:224)@0xb5dbfa10
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:183)@0xb5cfa8ce
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘2’ for queue: ‘weblogic.socket.Muxer'” prio=5 id=0x2100 pid=11322 blocked, daemon
— Blocked trying to get lock: java/lang/String@0x23282900[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.fatLockWaitOrSpin(Optimized Method)@0xb6955321
at jrockit/vm/Locks.fatLockReacquire(Optimized Method)@0xb223f5f1
at jrockit/vm/Locks.lockFat(Optimized Method)@0xb312a2f9
at jrockit/vm/Locks.monitorEnterSecondStage(Optimized Method)@0xb3126e44
at jrockit/vm/Locks.monitorEnter(Native Method)@0xb738b694
at weblogic/socket/PosixSocketMuxer.processSockets(PosixSocketMuxer.java:91)@0xb380039c
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:32)@0xb380033a
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:224)@0xb5dbfa10
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:183)@0xb5cfa8ce
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘JMS.TimerClientPool'” prio=5 id=0x2180 pid=11324 waiting, daemon
— Blocked trying to get lock: weblogic/kernel/ExecuteThreadManager@0x214ba060[thin lock]
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x21e9acf0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x21e9acf0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘1’ for queue: ‘JMS.TimerClientPool'” prio=5 id=0x2200 pid=11325 waiting, daemon
— Blocked trying to get lock: weblogic/kernel/ExecuteThreadManager@0x214ba060[thin lock]
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x2325c688[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x2325c688[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘2’ for queue: ‘JMS.TimerClientPool'” prio=5 id=0x2280 pid=11326 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x2325cc18[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x2325cc18[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘3’ for queue: ‘JMS.TimerClientPool'” prio=5 id=0x2300 pid=11327 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x2325d1a8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x2325d1a8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘4’ for queue: ‘JMS.TimerClientPool'” prio=5 id=0x2380 pid=11328 waiting, daemon
— Blocked trying to get lock: weblogic/kernel/ExecuteThreadManager@0x214ba060[thin lock]
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x2325d738[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x2325d738[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘5’ for queue: ‘JMS.TimerClientPool'” prio=5 id=0x2400 pid=11329 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x2325dcf8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x2325dcf8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“Thread-16” prio=5 id=0x2480 pid=23439 active, daemon
at jrockit/net/SocketNativeIO.read(Native Method)@0xb37ed2f0
at jrockit/net/SocketNativeIO.read(Unknown Source)@0xb37ed361
at java/net/AbstractSocketImpl$1.read(Optimized Method)@0xb312bf11
at jrockit/io/NativeIOInputStream.read(Optimized Method)@0xb2242179
at java/io/BufferedInputStream.fill(BufferedInputStream.java:183)@0xb68b98fd
at java/io/BufferedInputStream.read1(BufferedInputStream.java:222)@0xb73bebf5
at java/io/BufferedInputStream.read(BufferedInputStream.java:277)@0xb73beadb
^– Holding lock: java/io/BufferedInputStream@0x23d2ad80[thin lock]
at com/sun/jndi/ldap/Connection.run(Connection.java:780)@0xb2ef7d0c
at java/lang/Thread.run(Unknown Source)@0xb690a62f
at java/lang/Thread.startThreadFromVM(Optimized Method)@0xb312a12a
— End of stack trace
Blocked lock chains
===================
Chain 2:
“ExecuteThread: ‘2’ for queue: ‘weblogic.socket.Muxer'” (id: 0x00002100) waiting for java/lang/String@0x23282900 held by:
“ExecuteThread: ‘0’ for queue: ‘weblogic.socket.Muxer'” (id: 0x00002000) in chain 1
Open lock chains
================
Chain 1:
“ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” (id: 0x00002080) waiting for java/lang/String@0x23282900 held by:
“ExecuteThread: ‘0’ for queue: ‘weblogic.socket.Muxer'” (id: 0x00002000) (active)
Hi Eina,
High CPU usually causes Server hang, So we will have to first find out Which Thread is consuming High CPU. just by looking at the thread dump it is not possible to find out which Thread is consuming High CPU so we require the “Thread Dump” as well as the “top” command results to map the results.
You can refer to the following article to know what all things are required in order to investigate High CPU issues: http://middlewaremagic.com/weblogic/?p=4884
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
I am facing a problem, java.net.SocketException: Socket Closed
The Weblogic server showing up and running but the service is not available.Due to this i am doing bouncing every time . need your help here.please find the below exception
>
#### <> 11/12/27 21:44:32:947: (ter.ets.HttpChan
nel.doPost) [Warning] getReturnMapFromException returned null, ex = com.xxxx.ets.exception.EtsConfigurationException: Return Map not found for Name:
GeneralRpaFailure
>
#### 11/12/27 21:44:32:939: (er.rpa.RpaManager
.dispatch) [Warning] getReturnMapFromException returned null, ex = java.rmi.RemoteException: SOAP Fault:javax.xml.rpc.soap.SOAPFaultException: Failed to sen
d request
Detail:
java.net.SocketException: Socket Closed
at java.net.PlainSocketImpl.setOption(PlainSocketImpl.java:177)
at java.net.Socket.setTcpNoDelay(Socket.java:771)
at weblogic.net.http.HttpClient.openServer(HttpClient.java:305)
at weblogic.net.http.HttpClient.openServer(HttpClient.java:354)
at weblogic.net.http.HttpClient.<init>(HttpClient.java:139)
at weblogic.net.http.HttpClient.New(HttpClient.java:212)
at weblogic.net.http.HttpURLConnection.connect(HttpURLConnection.java:140)
at weblogic.net.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:189)
at weblogic.webservice.binding.http11.Http11ClientBinding.send(Http11ClientBinding.java:427)
at weblogic.webservice.core.handler.ClientHandler.handleRequest(ClientHandler.java:37)
at weblogic.webservice.core.HandlerChainImpl.handleRequest(HandlerChainImpl.java:143)
at weblogic.webservice.core.ClientDispatcher.send(ClientDispatcher.java:231)
at weblogic.webservice.core.ClientDispatcher.dispatch(ClientDispatcher.java:143)
at weblogic.webservice.core.DefaultOperation.invoke(DefaultOperation.java:471)
at weblogic.webservice.core.DefaultOperation.invoke(DefaultOperation.java:457)
at weblogic.webservice.core.rpc.StubImpl._invoke(StubImpl.java:303)
at com.prairiesys.IVBXML.RPA_PortType_Stub.getAddress(RPA_PortType_Stub.java:26)
at com.xxxx.rpa.prairie.RPAPrairieWSClient.getAddress(Lcom.prairiesys.www.IVBXML.GetAddress;)Lcom.prairiesys.www.IVBXML.GetAddressResponse;(Unk
nown Source)
at com.xxxx.rpa.cpa.CpaDispatcher.getAddress(Lcom.xxxx.rpa.mesg.RpaAdressRequestMessage;Ljava.lang.String;)Lcom.xxxx.rpa.mesg.Rpa
ResponseMessage;(Unknown Source)
at com.r.rpa.RpaManager.dispatch(Lcom.xxxx.ets.mesg.EtsRequestMessage;)Lcom.xxxx.ets.mesg.EtsResponseMessage;(Unknown Source)
at com.xxxx.ets.EtsRouter.dispatch(Lcom.xxxx.ets.mesg.EtsRequestMessage;)Lcom.xxxx.ets.mesg.EtsResponseMessage;(Unknown Source)
at com.xxxx.ets.HttpChannel.doPost(Ljavax.servlet.http.HttpServletRequest;Ljavax.servlet.http.HttpServletResponse;)V(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1077)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:465)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:348)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:7047)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3902)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2773)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
:
; nested exception is:
javax.xml.rpc.soap.SOAPFaultException: Failed to send request
>
#### <>
#### <>
#### <> <A stack trace associated with message 220048 follows:
java.net.SocketException: Socket Closed
at java.net.PlainSocketImpl.setOption(PlainSocketImpl.java:177)
at java.net.Socket.setTcpNoDelay(Socket.java:771)
at weblogic.net.http.HttpClient.openServer(HttpClient.java:305)
at weblogic.net.http.HttpClient.openServer(HttpClient.java:354)
at weblogic.net.http.HttpClient.(HttpClient.java:139)
at weblogic.net.http.HttpClient.New(HttpClient.java:212)
at weblogic.net.http.HttpURLConnection.connect(HttpURLConnection.java:140)
at weblogic.net.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:189)
at weblogic.webservice.binding.http11.Http11ClientBinding.send(Http11ClientBinding.java:427)
at weblogic.webservice.core.handler.ClientHandler.handleRequest(ClientHandler.java:37)
at weblogic.webservice.core.HandlerChainImpl.handleRequest(HandlerChainImpl.java:143)
at weblogic.webservice.core.ClientDispatcher.send(ClientDispatcher.java:231)
at weblogic.webservice.core.ClientDispatcher.dispatch(ClientDispatcher.java:143)
at weblogic.webservice.core.DefaultOperation.invoke(DefaultOperation.java:471)
at weblogic.webservice.core.DefaultOperation.invoke(DefaultOperation.java:457)
at weblogic.webservice.core.rpc.StubImpl._invoke(StubImpl.java:303)
at com.prairiesys.IVBXML.RPA_PortType_Stub.getAddress(RPA_PortType_Stub.java:26)
at com.xxxx.rpa.prairie.RPAPrairieWSClient.getAddress(Lcom.prairiesys.www.IVBXML.GetAddress;)Lcom.prairiesys.www.IVBXML.GetAddressResponse;(Unk
nown Source)
at com.xxxx.rpa.cpa.CpaDispatcher.getAddress(Lcom.xxxx.rpa.mesg.RpaAdressRequestMessage;Ljava.lang.String;)Lcom.xxxx.rpa.mesg.Rpa
ResponseMessage;(Unknown Source)
at com.xxxx.rpa.RpaManager.dispatch(Lcom.xxxx.ets.mesg.EtsRequestMessage;)Lcom.xxxx.ets.mesg.EtsResponseMessage;(Unknown Source)
at com.xxxx.ets.EtsRouter.dispatch(Lcom.xxxx.ets.mesg.EtsRequestMessage;)Lcom.xxxx.ets.mesg.EtsResponseMessage;(Unknown Source)
at com.xxxx.ets.HttpChannel.doPost(Ljavax.servlet.http.HttpServletRequest;Ljavax.servlet.http.HttpServletResponse;)V(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1077)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:465)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:348)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:7047)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3902)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2773)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
The above exception trace indicates some sort of N./W communication issue. Can you please check whether the service/WSDL address mentioned inside your code ” com.xxxx.rpa.prairie.RPAPrairieWSClient” method getAddress() is reachable or not when you are getting the above exception?
Also If you are using WebLogic 8.1 then enable the following JAVA_OPTION on server as well as on WebService client end to get more detailed informatoins of the WebService invocation failure: (For WebLogic 8.1 JAVA_OPTION) -Dweblogic.webservice.verbose=true
Hi Team,
Can you please provide me the scripts which automatically sends an email to us when the Weblogic servers are in Warning state or shutdown state. Thank you.
i am trying to find out the state of n number of message bridges which are configured in my domain. for that i wrote a wlst program. but while execution it failed. Please help me how to find out the state of the message bridges. Find the below wlst script.
connect(‘system’,’password’,’t3://nus690pb:14101′)
servers = domainRuntimeService.getServerRuntimes();
if (len(servers) > 0):
for server in servers:
messagingBridgeRuntimeEntry = server.getMessagingBridgeRuntimeEntry;
messageBridges = messagingBridgeRuntimeEntry.getMessageBridges();
for messageBridge in messageBridges:
destinations = messageBridge.getDestinations();
for destination in destinations:
print ‘ Name ‘ , desitnation.getName()
print ‘ State ‘ , destination.getState()
connect(‘weblogic’,’weblogic’,’t3://localhost:7001′)
servers = domainRuntimeService.getServerRuntimes();
if (len(servers) > 0):
for server in servers:
messagingBridgeRuntimeEntry = server.getMessagingBridgeRuntimeEntry();
messageBridges = messagingBridgeRuntimeEntry.getMessageBridges();
for messageBridge in messageBridges:
destinations = messageBridge.getDestinations();
for destination in destinations:
print ‘ Name ‘ , desitnation.getName()
print ‘ State ‘ , destination.getState()
As you said made the changes to the above script was failed to execute. Find the error log details below and provide me how to get the state of a message bridge using wlst.
C:Documents and Settingssiva_guddetiDesktop>wlst JMSBridgeRuntime.py
Welcome to WebLogic Server Administration Scripting Shell
Type help() for help on available commands
Connecting to t3://nus690pb:14101 with userid system …
Successfully connected to Admin Server ‘router_server’ that belongs to domain ‘r
outerdomain’.
Warning: An insecure protocol was used to connect to the
server. To ensure on-the-wire security, the SSL port or
Admin port should be used instead.
Problem invoking WLST – Traceback (innermost last):
File “C:Documents and Settingssiva_guddetiDesktopJMSBridgeRuntime.py”, lin
e 5, in ?
AttributeError: instance of ‘$Proxy5’ has no attribute ‘getMessagingBridgeRuntime’
Note that the ServerRuntimeMBean does not have a relation to the MessagingBridgeRuntimeMBean, so the server.getMessagingBridgeRuntimeEntry() will throw the error you are observing – AttributeError: instance of ‘$Proxy5′ has no attribute ‘getMessagingBridgeRuntime’
I am new to FMW. For single sign on we are using OAM which is configured with Web Server, the app… server remains on another machine. For each environment we are maintaining respective WebTier server( for dev it is having dev mw server which is configured with devwt server, for qa also having the same which is having their own configurations internally). Can I use only one webtier server which is configure with devmw & qamw, so that users can login from single point to multiple environments.
Can you please share some information on this?
Thanks,
Pavan.
Re-using Webtier/OAM for multiple environments that share same AD.
Hi,
I am trying to deploy a library that comes with weblogic, but getting the following error. It’s reporting that the library does not have the specification version. It’s not custom build library, I am not sure what’s wrong. pls help me out.
.Default (self-tuning)’> <Console encountered the following error weblogic.application.library.Li
braryDeploymentException: [J2EE:160144]Failed to register library Extension-Name: glassfish.el_2.2.0, Implementation-Version: 2.2.0: Library ca
nnot have Implementation-Version set, without also specifying its Specification-Version.
at weblogic.application.internal.library.LibraryDeployment.prepare(LibraryDeployment.java:49)
at weblogic.application.internal.DeploymentStateChecker.prepare(DeploymentStateChecker.java:154)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.prepare(AppContainerInvoker.java:60)
at weblogic.deploy.internal.targetserver.operations.ActivateOperation.createAndPrepareContainer(ActivateOperation.java:207)
at weblogic.deploy.internal.targetserver.operations.ActivateOperation.doPrepare(ActivateOperation.java:98)
at weblogic.deploy.internal.targetserver.operations.AbstractOperation.prepare(AbstractOperation.java:217)
at weblogic.deploy.internal.targetserver.DeploymentManager.handleDeploymentPrepare(DeploymentManager.java:747)
at weblogic.deploy.internal.targetserver.DeploymentManager.prepareDeploymentList(DeploymentManager.java:1216)
at weblogic.deploy.internal.targetserver.DeploymentManager.handlePrepare(DeploymentManager.java:250)
at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.prepare(DeploymentServiceDispatcher.java:159)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doPrepareCallback(DeploymentReceiverCallbackDelive
rer.java:171)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.access$000(DeploymentReceiverCallbackDeliverer.jav
a:13)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer$1.run(DeploymentReceiverCallbackDeliverer.java:46)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Hi Rene,
Thanks for your quick response. No, it does not have the specification version. It does not make any sense to me why weblogic wants the jar file to be edited it’s deployed. Because it’s supplied with Weblogic and it should not expect users to edit it if it is mandatory to have the specification version. Hopefully Oracle would consider including the specification version in the jar file in future. Again, thanks for your help.
—————
Implementation-Title: Expression Language 2.1 for JSP 2.2 Wed Jul 1 01
:34:54 PDT 2009
Implementation-Version: 2.2.0
Source-Repository-Change-Id: 1234052
—————–
Hi, I am getting the below error. (note: I replaced the hostname and IPs with dummy values). It appears that he certificate contained the hostname with domain name, but the weblogic check is returning only the hostname with out the domain name. Can you please help me out? I know I can disable the hostname verifier, but I am looking for a solution without disabling the hostname varifier.
————–
Certificate chain received from test1 – 10.00.00.01 failed hostname verification check. Certificate contained test1.raj.com but check expected test1
———
Rene, Thanks for your quick reply. I already disabled the hostname verification and it’s working fine. But as I mentioned in my post, I am looking for a solution without disabling the hostname verification. Do you have any idea on what would be wrong with the weblogic check?
Recently i face the below issue. once i configured the weblogic resoures using the lock and edit button and saved it. After that when i click the activate button the server going down with the below exception. This happens frequently. i am using the weblogic 9.2 MP3 version. Please help me how to resolve the issue.
12 10:31:33 PM EST>
<There was an error while running a lifecycle stage :: Lifecycle: UIControl.render :: for the control :: null ::.
com.bea.netuix.nf.UIControlException: com.bea.portlet.adapter.scopedcontent.ActionLookupFailedException: java.lang.NoSuchMethodError: org.jdom.Element.addContent(Lorg/jdom/Element;)Lorg/jdom/Element;
at com.bea.netuix.servlets.controls.content.NetuiContent.checkPreRenderExceptions(NetuiContent.java:400)
at com.bea.netuix.servlets.controls.content.NetuiContent.beginRender(NetuiContent.java:339)
at com.bea.netuix.nf.ControlLifecycle$7.visit(ControlLifecycle.java:486)
at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:530)
at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:541)
Truncated. see log file for complete stacktrace
com.bea.portlet.adapter.scopedcontent.ActionLookupFailedException: java.lang.NoSuchMethodError: org.jdom.Element.addContent(Lorg/jdom/Element;)Lorg/jdom/Element;
at com.bea.portlet.adapter.scopedcontent.ScopedContentCommonSupport.executeAction(ScopedContentCommonSupport.java:699)
at com.bea.portlet.adapter.scopedcontent.ScopedContentCommonSupport.processActionInternal(ScopedContentCommonSupport.java:144)
at com.bea.portlet.adapter.scopedcontent.StrutsStubImpl.processAction(StrutsStubImpl.java:76)
at com.bea.portlet.adapter.NetuiActionHandler.raiseScopedAction(NetuiActionHandler.java:99)
at com.bea.netuix.servlets.controls.content.NetuiContent.raiseScopedAction(NetuiContent.java:180)
Truncated. see log file for complete stacktrace
java.lang.NoSuchMethodError: org.jdom.Element.addContent(Lorg/jdom/Element;)Lorg/jdom/Element;
at com.bea.console.preferences.spi.FilePreferencesProvider.writeUserPreferences(FilePreferencesProvider.java:490)
at com.bea.console.preferences.spi.FilePreferencesProvider.writeToFile(FilePreferencesProvider.java:409)
at com.bea.console.preferences.spi.FilePreferencesProvider.store(FilePreferencesProvider.java:221)
at com.bea.portlet.prefs.spi.PreferencePersistenceDelegate.storePreferences(PreferencePersistenceDelegate.java:120)
at com.bea.portlet.prefs.spi.AbstractPortletPreferences.internalStore(AbstractPortletPreferences.java:459)
Truncated. see log file for complete stacktrace
Our team has a scenario with BPEL composites migration from one instance to another e.g. from Dev to Test to UAT, etc . When the jar, that we(admins) migrate from one instance to another, has references to first instance (Dev)/previous instance , it creates an issue.
Under this situation, if the first/previous instance is down due to some reason and if the deployed composite in current instance has references to previous instance, the composite doesn’t work as expected.
Practices followed :
Config plans are used during migration from first instance to the next instance , which ‘searches & replaces’ the references to previous instances.
Problem faced :
In spite of strict standard that is being followed, it is human that there are slips and misses. And during such scenarios, when the composites dont work as expected, all hell breaks loose !
Partial Solution :
A partial solution that we found could restrict references to other instances, and could give a clear indication while composite deployment itself, is setting up ‘ Connection Filter’ under Domain > Security > Filter.
This (when configured in first/previous instance) works as access blocking rule i.e. any requests from current instance to previous instance will be blocked ‘at’ previous instance.
The catch here is , when a previous instance is on the same ‘IP’ as the current instance, setting up this connection filter rule doesnt solve the purpose.
Question :
Is there any other method to restrict ‘outgoing requests from current instance’ than blocking incoming requests in previous instances ?
I am seeing so many .tmp files sitting in /tmp folder and not cleaning up by their own. Can anyone tell me why these files are generating and siting in /tmp and also please let me know how to get rid of these files.
EX: /tmp/1358967927859726047624544989663.tmp
using jrockit_160_29_D1.2.0-10 (weblogic -10.3.6)
Before I used to see .jfr files created in /tmp.. giving -XX:-FlightRecorder in arguments solved that problem, but still seeing.tmp files generating.
I’m wondering if anyone has ever faced the issue where a WLS plugin mod_wl.so is setup on Apache and the Apache instance is hosted in a DMZ where SSL is terminated. The problem comes where I have turned off SSL Listen port on the WLS server which is not in the DMZ and traffic between the Apache WebLogic Plugin and the WebLogic Server in http. SSL is setup on the Apache server and the secure icon is working great until the WLS server uses something like a call to google maps API on https. Now I have mixed mode content and my nice green cert icon gets a red slash. I would prefer not to use the http version of the google API, but that would seemingly fix the cert issue. Does anyone know the real solution? You must be able to terminate SSL at the Plug-in so that traffic can be inspected by an IDS. Maybe I’m missing something. Any comments or suggestions are welcome.
I may have posted too soon. The way to do this is likely to be a simple code change to make sure the call to the google API comes back in through the https Apache with WLS Plug-in host and not back to the application on the app Tier. All communication that leaves our network must come back in through the DMZ, it’s that simple.
Just needed to talk (write) this one out.
Thanks! Comments or suggestions are still very much welcome.
It seems that my attempt to fix this did not work. Does anyone have advice?
The situation is a N-Tier environment where (WEB – Apache with mod_wl.so proxying to WLS, Application – Weblogic Server, and DB – Oracle instance) SSL is terminated on the Web Tier at Apache and passed on using mod_wl.so to the Application Tier running WebLogic 10.3.3. The Application then makes a call to google API for calculating distance betweeen two points using the https URL. Because I terminate SSL on the Web Tier we now have mixed content and a nasty red x on my https in the URL making users frightened.
The URL the user normally sees is https://server/application where server is the Apache server. How do I fix this? Am I missing something on the WLS setup or is it just fundamental that if the site is http at this point and wants to call https mixed content will occur. There has to be a solution without using google http APIs. Any advice?
I have found the solution to the above problem where SSL is terminated at the Web Tier and the Application Tier makes a request to an https URL. You need to get the version of the Apache WLS plugin called 1.1 which does not ship with WLS. Then in your httpd.conf simply set the directive:
WLProxySSL ON
Under certain conditions on of which is descrbed above the directive does the following – When WLProxySSL is set to ON, the location header returned to the client from WebLogic Server specifies the HTTPS protocol.
Hope this helps someone. This website is a great resource! Thank you Magic Team.
I was looking for some information on Synchronous and Asynchronous data sources in Weblogic. I googled it, but couldn’t get much information. It will be great if you could shed some light on this.
August 3rd, 2010 on 11:56 am
Hi Vinoth,
Can you make it clear on development vs production mode with respective Lock feature ?
What do you mean by saying like the “Lock&Edit” feature NOT available in production?
I’m working in production mode , and i can use this feature without any problem.
I click on Lock&Edit from console , apply my changes , and then finally release it! , so that works fine for me.
Let me know if anything wrong in my assumption , and awaiting for your valauble points for me.
Thanks Much
August 3rd, 2010 on 1:10 pm
Hi Arun,
From WLS10.3 onwards enabling and disabling feature of Lock & Edit button is added.
Login to AdminConsole—> Click on (Preference) link —-> User preferences (Tab) —->Bottom off the page you will see a check box: Automatically Acquire Lock and Activate Changes
Try to check and Uncheck that Field in production and development Mode…and then check if you see any differences in both scenarios?
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
August 3rd, 2010 on 1:19 pm
Thanks Jay
I got the difference in both cases , thanks for clarifying!
–Arun
August 4th, 2010 on 10:49 am
That sounds good to me..
Thanks much Vinoth for your help!
I’ve one question , but not sure whether i can ask here or not..
A very important part of WebLogic Administration is to do with resolving and troubleshooting issues most notably in Production.
Our value is determined by how well we diagnose the problems, bring the services up, determine the root cause of the issue and prevent it from happening in future.
The following are typical reports from users for issues
* The application does not respond and browser just clocks
* Users receiving a page that says Internal Server intermittently
* Users receiving pages that have “404” – Page not found intermittently
* Users getting Page cannot be displayed errors intermittently
* The application is being very slow than usual
What are basic/common approaches to these types of problems…?
–Arun
August 4th, 2010 on 12:12 pm
Hi Arun,
There is no hard runles to investigate these kind of issues because every time for these kind of issues we find different root causes…. It can be solved by analyzing the Logs/out/debugging only. How to start debugging these issue …is simply “Look at the server logs…if you dont find much informations then enable debug flags to get more informations….etc”
* The application does not respond and browser just clocks …Take 5-6 Thread Dumps in the interval of 10-12 seconds each… and analyze them.
* Users receiving a page that says Internal Server intermittently Internal Server Error or Error Code 500 in the browser means you need to look into the server Logs to find out the root cause of failure
* Users receiving pages that have “404″ – Page not found intermittently Check the “access.log” request is reaching to server or not for the correct resource…Check the URL is correct or not? Check the Server Logs for any kind of Application related WARNNING or Error Message. You can even check the application logs (if you have any).
* Users getting Page cannot be displayed errors intermittently. Check the “access.log” request is reaching to server or not for the correct resource…Check the URL is correct or not? Check the Server Logs for any kind of Application related WARNNING or Error Message. You can even check the application logs (if you have any).
* The application is being very slow than usual. Take 5-6 Thread Dumps in the interval of 10-12 seconds each… and analyze them.
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
August 4th, 2010 on 12:20 pm
Great!
Thanks , let me follow your steps next time when i face these issues again , and still if i had issues I’ll update you.
Thanks again Jay.
August 4th, 2010 on 5:58 pm
Thanks for the information.
I’ll let you know incase of any issues in my app.
Thanks
Arun
August 5th, 2010 on 7:21 pm
Hi,
How can we decide when to use JMS Queues( or JMS Messaging Bridges) and MQ for interacting with other systems.
It would be better if pros & cons are provided.
Cheers,
Surya.
August 18th, 2010 on 1:50 am
Hi Jay
I will first say Thanks to you and faisal for sharing such a good info about weblogic . This site will helping to everyone who intrested in weblogic administration.
I need your help to understand Thread Dump(TD).
As per your post there are many ways we can take thread dumps, But how we can understand it.
for eg I am pasting here some part of TD.
requesting you to please provide your inputs for same.
=======================================================
===== FULL THREAD DUMP ===============
Fri Aug 13 10:24:24 2010
BEA JRockit(R) R27.6.0-50_o_CR373585-101992-1.5.0_15-20080813-1049-linux-ia32
“Main Thread” id=1 idx=0x4 tid=28104 prio=5 alive, in native, waiting
— Waiting for notification on: weblogic/t3/srvr/T3Srvr@0x86b9a98[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:474)
at weblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:730)
^– Lock released while waiting: weblogic/t3/srvr/T3Srvr@0x86b9a98[fat lock]
at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:380)
at weblogic/Server.main(Server.java:67)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
— end of trace
“(Signal Handler)” id=2 idx=0x8 tid=28105 prio=5 alive, in native, daemon
“(GC Main Thread)” id=3 idx=0xc tid=28106 prio=5 alive, in native, native_waiting, daemon
“(GC Worker Thread 1)” id=? idx=0x10 tid=28107 prio=5 alive, in native, daemon
“(GC Worker Thread 2)” id=? idx=0x14 tid=28108 prio=5 alive, in native, daemon
“(GC Worker Thread 3)” id=? idx=0x18 tid=28109 prio=5 alive, in native, daemon
“(GC Worker Thread 4)” id=? idx=0x1c tid=28110 prio=5 alive, in native, daemon
“(GC Worker Thread 5)” id=? idx=0x20 tid=28111 prio=5 alive, in native, daemon
“(GC Worker Thread 6)” id=? idx=0x24 tid=28112 prio=5 alive, in native, daemon
“(GC Worker Thread 7)” id=? idx=0x28 tid=28113 prio=5 alive, in native, daemon
“(GC Worker Thread 8)” id=? idx=0x2c tid=28114 prio=5 alive, in native, daemon
“(Code Generation Thread 1)” id=4 idx=0x30 tid=28115 prio=5 alive, in native, native_waiting, daemon
“(Code Optimization Thread 1)” id=5 idx=0x34 tid=28116 prio=5 alive, in native, native_waiting, daemon
“(VM Periodic Task)” id=6 idx=0x38 tid=28117 prio=10 alive, in native, daemon
“Finalizer” id=7 idx=0x3c tid=28118 prio=8 alive, in native, native_waiting, daemon
at jrockit/memory/Finalizer.waitForFinalizees([Ljava/lang/Object;)I(Native Method)
at jrockit/memory/Finalizer.access$500(Finalizer.java:12)
at jrockit/memory/Finalizer$4.run(Finalizer.java:159)
at java/lang/Thread.run(Thread.java:595)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
— end of trace
“Reference Handler” id=8 idx=0x40 tid=28119 prio=10 alive, in native, native_waiting, daemon
at java/lang/ref/Reference.waitForActivatedQueue()Ljava/lang/ref/Reference;(Native Method)
at java/lang/ref/Reference.access$100(Reference.java:11)
at java/lang/ref/Reference$ReferenceHandler.run(Reference.java:79)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
— end of trace
“(Sensor Event Thread)” id=9 idx=0x44 tid=28120 prio=5 alive, in native, daemon
“Timer-0” id=12 idx=0x48 tid=28121 prio=5 alive, in native, waiting, daemon
— Waiting for notification on: java/util/TaskQueue@0x8597528[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:2250)[inlined]
at java/lang/Object.wait(Object.java:474)[optimized]
at java/util/TimerThread.mainLoop(Timer.java:483)
^– Lock released while waiting: java/util/TaskQueue@0x8597528[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
— end of trace
“Timer-1” id=13 idx=0x4c tid=28165 prio=5 alive, in native, waiting, daemon
— Waiting for notification on: java/util/TaskQueue@0xb084b20[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at java/util/TimerThread.mainLoop(Timer.java:509)
^– Lock released while waiting: java/util/TaskQueue@0xb084b20[fat lock]
at java/util/TimerThread.run(Timer.java:462)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
— end of trace
“[STANDBY] ExecuteThread: ‘0’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=14 idx=0x50 tid=28166 prio=5 alive, in native, waiting, daemon
— Waiting for notification on: weblogic/work/ExecuteThread@0xa5d7178[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:2250)[inlined]
at java/lang/Object.wait(Object.java:474)[inlined]
at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:165)[optimized]
^– Lock released while waiting: weblogic/work/ExecuteThread@0xa5d7178[fat lock]
at weblogic/work/ExecuteThread.run(ExecuteThread.java:186)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
— end of trace
“weblogic.time.TimeEventGenerator” id=15 idx=0x54 tid=28167 prio=9 alive, in native, waiting, daemon
— Waiting for notification on: weblogic/time/common/internal/TimeTable@0xa9ba5b0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at weblogic/time/common/internal/TimeTable.snooze(TimeTable.java:286)
^– Lock released while waiting: weblogic/time/common/internal/TimeTable@0xa9ba5b0[fat lock]
at weblogic/time/common/internal/TimeEventGenerator.run(TimeEventGenerator.java:117)
at java/lang/Thread.run(Thread.java:595)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
— end of trace
“JMAPI event thread” id=16 idx=0x58 tid=28168 prio=5 alive, in native, native_waiting, daemon
“weblogic.timers.TimerThread” id=17 idx=0x5c tid=28169 prio=9 alive, in native, waiting, daemon
— Waiting for notification on: weblogic/timers/internal/TimerThread@0xac0efd0[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at weblogic/timers/internal/TimerThread$Thread.run(TimerThread.java:260)
^– Lock released while waiting: weblogic/timers/internal/TimerThread@0xac0efd0[fat lock]
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
— end of trace
“[ACTIVE] ExecuteThread: ‘1’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=18 idx=0x60 tid=28170 prio=5 alive, in native, daemon
at java/net/PlainDatagramSocketImpl.receive0(Ljava/net/DatagramPacket;)V(Native Method)
^– Holding lock: java/net/PlainDatagramSocketImpl@0xa2d3f70[recursive]
at java/net/PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)
^– Holding lock: java/net/PlainDatagramSocketImpl@0xa2d3f70[thin lock]
at java/net/DatagramSocket.receive(DatagramSocket.java:712)
^– Holding lock: java/net/DatagramPacket@0x21788670[thin lock]
^– Holding lock: java/net/MulticastSocket@0xa2d3f38[thin lock]
at weblogic/cluster/FragmentSocket.receive(FragmentSocket.java:202)
at weblogic/cluster/MulticastManager.run(MulticastManager.java:400)
at weblogic/work/ExecuteThread.execute(ExecuteThread.java:209)
at weblogic/work/ExecuteThread.run(ExecuteThread.java:181)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
— end of trace
“weblogic.store.WLS_DIAGNOSTICS” id=19 idx=0x64 tid=30093 prio=10 alive, in native, parked, daemon
at jrockit/vm/Locks.park0(J)V(Native Method)
at jrockit/vm/Locks.park(Locks.java:2506)
at sun/misc/Unsafe.park(ZJ)V(Native Method)
at java/util/concurrent/locks/LockSupport.park(LockSupport.java:118)
at java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1841)
at java/util/concurrent/LinkedBlockingQueue.take(LinkedBlockingQueue.java:359)
at weblogic/utils/concurrent/JDK15ConcurrentBlockingQueue.take(JDK15ConcurrentBlockingQueue.java:89)
at weblogic/store/internal/PersistentStoreImpl.getOutstandingWork(PersistentStoreImpl.java:570)
at weblogic/store/internal/PersistentStoreImpl.run(PersistentStoreImpl.java:618)
at java/lang/Thread.run(Thread.java:595)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
— end of trace
“ExecuteThread: ‘0’ for queue: ‘weblogic.socket.Muxer'” id=20 idx=0x68 tid=30096 prio=5 alive, in native, blocked, daemon
— Blocked trying to get lock: weblogic/socket/PosixSocketMuxer$1@0xb0ef598[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1674)[inlined]
at jrockit/vm/Locks.fatLockReacquire(Locks.java:1912)[optimized]
at jrockit/vm/Locks.lockFat(Locks.java:1780)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1311)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1258)[optimized]
at jrockit/vm/Locks.monitorEnter(Locks.java:2455)[optimized]
at weblogic/socket/PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
— end of trace
“ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” id=21 idx=0x6c tid=30097 prio=5 alive, in native, blocked, daemon
— Blocked trying to get lock: weblogic/socket/PosixSocketMuxer$1@0xb0ef598[fat lock]
at jrockit/vm/Threads.waitForUnblockSignal()V(Native Method)
at jrockit/vm/Locks.fatLockBlockOrSpin(Locks.java:1674)[optimized]
at jrockit/vm/Locks.lockFat(Locks.java:1775)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1311)[optimized]
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1258)[optimized]
at jrockit/vm/Locks.monitorEnter(Locks.java:2455)[optimized]
at weblogic/socket/PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93)
at weblogic/socket/SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:145)
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:117)
at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
— end of trace
“ExecuteThread: ‘2’ for queue: ‘weblogic.socket.Muxer'” id=22 idx=0x70 tid=30098 prio=5 alive, in native, daemon
at weblogic/socket/PosixSocketMuxer.poll([Lweblogic/socket/PosixSocketInfo$FdStruct;I)Z(Native Method)
at weblogic/socket/PosixSocketMuxer.processSockets(PosixSocketMuxer.java:102)
^– Holding lock: weblogic/socket/PosixSocketMuxer$1@0xb0ef598[fat lock]
Chain 2:
“ExecuteThread: ‘2’ for queue: ‘weblogic.socket.Muxer'” id=22 idx=0x70 tid=30098 waiting for weblogic/socket/PosixSocketMuxer$1@0xb0ef598 held by:
“ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” id=21 idx=0x6c tid=30097 in chain 1
Chain 3:
“ExecuteThread: ‘3’ for queue: ‘weblogic.socket.Muxer'” id=23 idx=0x74 tid=30099 waiting for weblogic/socket/PosixSocketMuxer$1@0xb0ef598 held by:
“ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” id=21 idx=0x6c tid=30097 in chain 1
Chain 4:
“ExecuteThread: ‘4’ for queue: ‘weblogic.socket.Muxer'” id=24 idx=0x78 tid=30100 waiting for weblogic/socket/PosixSocketMuxer$1@0xb0ef598 held by:
“ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” id=21 idx=0x6c tid=30097 in chain 1
Chain 5:
“ExecuteThread: ‘5’ for queue: ‘weblogic.socket.Muxer'” id=25 idx=0x7c tid=30101 waiting for weblogic/socket/PosixSocketMuxer$1@0xb0ef598 held by:
“ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” id=21 idx=0x6c tid=30097 in chain 1
Chain 6:
“ExecuteThread: ‘6’ for queue: ‘weblogic.socket.Muxer'” id=26 idx=0x80 tid=30102 waiting for weblogic/socket/PosixSocketMuxer$1@0xb0ef598 held by:
“ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” id=21 idx=0x6c tid=30097 in chain 1
Chain 7:
“ExecuteThread: ‘7’ for queue: ‘weblogic.socket.Muxer'” id=27 idx=0x84 tid=30103 waiting for weblogic/socket/PosixSocketMuxer$1@0xb0ef598 held by:
“ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” id=21 idx=0x6c tid=30097 in chain 1
Chain 8:
“ExecuteThread: ‘8’ for queue: ‘weblogic.socket.Muxer'” id=28 idx=0x88 tid=30104 waiting for weblogic/socket/PosixSocketMuxer$1@0xb0ef598 held by:
“ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” id=21 idx=0x6c tid=30097 in chain 1
Open lock chains
================
Chain 1:
“ExecuteThread: ‘0’ for queue: ‘weblogic.socket.Muxer'” id=20 idx=0x68 tid=30096 waiting for weblogic/socket/PosixSocketMuxer$1@0xb0ef598 held by:
“ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” id=21 idx=0x6c tid=30097 (active)
===== END OF THREAD DUMP ===============
In summary of TD tid=30097 is repeated many times.
why is it so ?
how above TD will help to investigate.
Thanks
Ravi
August 19th, 2010 on 11:42 am
Hi Ravi,
Sorry for delay in response. I donot see anything Unusual in the Above Thread Dump. Threads are locking different objects and releasing them as well…which is normal. Apart from that regarding tid=30097
“ExecuteThread: ‘1′ for queue: ‘weblogic.socket.Muxer’” id=21 idx=0×6c tid=30097 (active)
I can see that from weblogic.socket.Muxer Queue the Execute thread 1 is Active …which is normal. We dont expect that all the Muxer threads will be active at the same time….Some of them must be in the waiting state…
What SocketMuxers Do ?
The socket Muxer manages the server’s existing socket connections.
It first determines which sockets have incoming requests waiting to be processed. It then reads enough data to determine the protocol and dispatches the socket to an appropriate runtime layer based on the protocol.In the runtime layer, the socket muxer threads determine which execute thread queue to be used and delegates the request accordingly.
Weblogic has two versions of the socket muxer, one is the Java version and the other uses a native library which makes better use of operating system calls. Native Version — The Enable Native IO checkbox on the server’s configuration settings tells the server which version to use. This is ON by default for most platforms.
Analyzing One thread dump i not very useful…We should always collect at least 5-6 Thread Dumps taken in the interval of 9-10 Seconds each.
Ravi Also can u please specify which kind of issue you are facing and Like of Stuck Thread or Slow Response….Because from the Thread Dump i cannot see any such issue.
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
October 19th, 2010 on 7:41 pm
Hi all,
Hopefully someone can shed some light on my issue, I can’t believe I am the first to see this.
I have a working set of build scripts for creation and configuration of weblogic domains. Previously I have been able to put default passwords, DB passwords etc into a set of property files for the scripts to configure against.
New security regulations mean that I now need to either prompt the user building the domains to input a password (this cannot be echoed in anyway), encrypt it pre-configuration or have it as a manual task post installation. (The latter being the least desirable option)
I downloaded jython and got hold of getpass as trawling the internet this looked like a good viable option. Unfortunately when getpass is invoked I get
“Warning: Problem with getpass. Passwords may be echoed”)
….and sure enough they are.
Has anyone had to do anything like this before? Essentially I am trying to simply mask raw_input.
I fail to believe I am the first that’s had this issue. I am using WLS11Gr1 on jrockit 1.6.0_14 (Windows) and Sun 1.5.0_23 (Solaris)
Any help or suggestions would be greatly appreciated.
Many thanks!
Dave
October 20th, 2010 on 2:44 pm
Spot on! If you have any joy then it’d be seriously appreciated; someone must have attempted this somewhere before!
Thanks
November 15th, 2010 on 3:50 pm
6 weeks later 🙂
Thanks to Skott over at Daniweb 🙂
November 15th, 2010 on 5:14 pm
Hi ds2000,
Welcome to Middleware Magic.
We would like to thank you for sharing this script.
However I found out the link where Skott had shared this script, hence would like to share that link with other too.
Masking raw_input / disable screen output
http://www.daniweb.com/forums/post1385386.html#post1385386
Regards,
Ravish Mody
November 23rd, 2010 on 11:09 pm
Hi Jey/Ravish,
I am setting Oracle Fusion Middleware env on a RHEL 5.3.
I have done with the installation of Oracle Weblogic Server and try to setup the identity management suite. I have the following queries.
1) I am able to install the Oracle Weblogic with root. But for others like IDM Suite, Oracle webtier utilities the installation failed saying “Installation with root is not allowed.” I am not sure why
2) After I have created the reqd users and groups I am able to start the IDM installation but I got the following pre-req package checks failed.
Checking for compat-libstdc++-33-3.2.3-x86_64; Not found. Failed <<<<
Checking for compat-libstdc++-33-3.2.3-i386; Not found. Failed <<<<
Checking for elfutils-libelf-devel-0.125; Not found. Failed <<<<
Checking for gcc-4.1.1; Not found. Failed <<<<
Checking for gcc-c++-4.1.1; Not found. Failed <<<<
Checking for glibc-devel-2.5-x86_64; Not found. Failed <<<<
Checking for glibc-devel-2.5-12-i386; Not found. Failed <<<<
Checking for libaio-devel-0.3.106; Not found. Failed <<<<
Checking for libstdc++-devel-4.1.1; Not found. Failed <<<<
Checking for sysstat-7.0.0; Not found. Failed <<<<
Check complete. The overall result of this check is: Failed <<<<
How I can get and Install the libs for which the check has failed above?
Thanks,
bobby
December 13th, 2010 on 12:43 pm
Hi Jay,
How to configure the Managed Server log in the Default path to our configured path.
Domainname -> Environment -> Servers -> MS1 -> Logging ->
initial you have create the directory in your configuration log file path
#cd /oracle
#mkdir adf_log
Default path for Managed server log:
Log file name: logs/MS1.log
The managed server default log path is ” /oracle/Middleware/user_projects/domain/domainname/servers/MS1/logs/ ”
Changed into our configured path:
Log file name: /oracle/adf_log/
you have configure the samba service into the newly created adf_log directory to provide read permisson. So all the Developer can see the log file in the Windows environment.
#cd /etc/samba/
#vi smb.conf
security = share
[adf_log]
path = /oracle/adf_log
guest ok = yes
The “Limit number of retained files” default option is unmarked.
But you have to limit the log file to store the hard disk. Because in our dev , test environment it occupy more than 30 GB space. So to avoid these storage in the Weblogic Application Server side.
Follow the below steps:-
Limit number of retained files –> mark
Files to retain: –> 50 ( it take 50 backup of MS1 log )
Log file rotation directory: –> /oracle/adf_log/backup
you have to create the “backup” directory inside the adf_log path.
#cd /oracle/adf_log
#mkdir backup
So you have to limit the Managed server log file upto 50 * 5 MB = 250 MB.
The main advantage is you have to see the latest file available in the “/oracle/adf_log/ “path
The previous log file are available in the ” /oracle/adf_log/backup/ “path. It store the backup of 50 managed server MS1 logs.
The same steps you have to configure for every managed server. To avoid the memory area in the Weblogic Application Server.
Regards,
S.Vinoth Babu
April 16th, 2011 on 10:24 am
First I wanted to thank you for putting together such a fantastic resource! This blog is immensely useful!
—
Issue: WLST Help
OS: RedHat Enterprise Linux 5 – 32bit
WLS: 10.3.3
JDK: jrockit_160_05_R27.6.2-20 , Xms1024 Xmx1024
I’m looking to create some WLST scripts which stop a set of servers, then start them again. These particular servers tend to come up in Admin mode because of connectivity issues with remote databases. I’m not having any luck getting these to resume from Admin mode. Does anyone have any suggestions?
April 16th, 2011 on 12:06 pm
Hi Chris,
We published an Article based on your scenario…Please have a look :
http://middlewaremagic.com/weblogic/?p=6407
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
May 9th, 2011 on 3:48 pm
Hi Team,
I noticed the below warning in the WLS log file.
<RuntimeException thrown by rmi server: weblogic.management.intern
al.RemoteMBeanServerImpl.getAttribute(Ljavax.management.ObjectName;Ljava.lang.String;)
java.lang.IllegalArgumentException: Object name for an MBean can not be null.
java.lang.IllegalArgumentException: Object name for an MBean can not be null
Can you please help me to give a fix for this. Thanks.
Rajendra
May 9th, 2011 on 6:22 pm
Hi Rajendra,
There is no specific reason for this WARNING. Investigation on this will require more detailed and in depth analysis of all the logs and the configuration …as well as what happed at the time when the WARNNING was occurred like was there any recent changes … We would recommend you to please Open A Oracle WebLogic Support ticket as well as additionally post the same query on Oracle Forums:
http://forums.oracle.com/forums/forum.jspa?forumID=570 To get more quick and accurate results.
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
July 26th, 2011 on 8:57 pm
Hi Jay,
I am new to Middleware Magic and not sure how to create a new post as I dont even seem to have access to search which was available earlier. I wanted to find out about Java process eating my CPU resources (Weblogic server with JRockit on Linux). What are some of the basic troubleshooting tips to find out whats causing the java process to tale almost 100 % CPU?
Apologies if this is not the right way for posting any new issues or queries.
Regards,
Bis
August 3rd, 2011 on 12:05 pm
Hi Guys,
Can someone help me in understanding the concept “Deployment Order” ?
To my knowledge i can brief certain things like –
1. The default deployment order is 100
2. Usually the order the way deployments works –
(a) JDBC
(b) JMS
(c) J2EE libraries , and optional packages if any
(d) Web applications and actual applications
(e) Startup classes
In my project i observed that one of the my application deployment had the order “100” , and some other with “161”
I just want to understand like –
1) Deployment order value 100 means ?
2) And what’s the value 161 says then ?
3) How to differentiate both?
4) Or purpose/use of this deployment order.
Thanks in advance
September 14th, 2011 on 2:33 am
Hi Jay,
I have the following error in the logs. Could you please suggest. What is the Socket IP and port that it is pointing to, i am not aware of that IP:port, whats that exactly?
#### <> <IOException occurred on socket: Socket[addr=/10.10.175.14,port=2096,localport=7017]
java.net.SocketException: Connection refused.
java.net.SocketException: Connection refused
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at weblogic.socket.SocketMuxer.readReadySocketOnce(SocketMuxer.java:842)
at weblogic.socket.SocketMuxer.readReadySocket(SocketMuxer.java:818)
at weblogic.socket.DevPollSocketMuxer.processSockets(DevPollSocketMuxer.java:120)
at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:42)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
Thank you,
Manish
September 27th, 2011 on 8:32 pm
Hi Jay,
1.Could you please explain why the XA transaction timeout should be greater than or equal to JTA (Global WL) timeout?
2.What is JTA timeout, XA timeout and EJB/MDB timeout? And how they have to be set? I mean which one should be greater than other?
Thank you,
Manish
September 27th, 2011 on 9:44 pm
Hi Manishy,
This is true for every transaction, That the application level liek ejbs/jms/mdb or Jdbc level transactions timeout value should be greater than or equal to the global WebLogic Server transaction timeout. Configuration settings for JTA (transactions) are applicable at the domain level. This means that configuration attribute settings apply to all servers within a domain.
For EJBs you can define the transaction timeout by using “weblogic-ejb-jar.xml” file tag “trans-timeout-seconds” http://download.oracle.com/docs/cd/E11035_01/wls100/ejb/DDreference-ejb-jar.html#wp1116332
“trans-timeout-seconds” should be greater than the Domain level Global transaction timeout duration.
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
October 8th, 2011 on 1:57 am
Hi Jay,
Thanks for the info.
I was monitoring the Heap Size and i observed that the Heap is growing to 100% and after Full GC it is settling down to 35-40% , could you please explain this behavior and i welcome your suggestions if you have any. Is it normal that the Heap settling down to 40% of its capacity? Here is an output of jstat that i took for every 10 minutes.
S0 S1 E O P YGC YGCT FGC FGCT GCT
0.00 99.94 79.71 60.76 97.11 45 6.620 0 0.000 6.620
0.00 82.19 82.81 76.25 97.77 73 15.281 0 0.000 15.281
10.47 0.00 52.29 84.19 98.88 86 18.600 0 0.000 18.600
31.04 0.00 36.31 88.70 99.29 106 21.773 0 0.000 21.773
26.61 0.00 66.35 94.90 99.82 128 26.492 0 0.000 26.492
0.00 99.96 42.14 46.33 49.24 149 32.820 1 7.294 40.114
33.62 0.00 39.03 48.52 49.47 164 35.088 1 7.294 42.381
0.00 86.70 95.99 51.93 49.64 183 38.241 1 7.294 45.534
0.00 99.99 28.78 62.55 49.90 203 43.577 1 7.294 50.871
37.49 0.00 69.00 65.38 50.04 216 46.585 1 7.294 53.879
0.00 99.76 61.33 72.21 50.27 239 49.508 1 7.294 56.802
100.00 0.00 43.33 82.31 50.44 268 51.944 1 7.294 59.238
0.00 72.73 34.29 98.09 50.55 297 58.759 1 7.294 66.053
34.75 0.00 51.93 36.49 44.42 312 61.483 2 12.694 74.176
0.00 99.97 95.84 44.46 49.80 323 63.397 2 12.694 76.091
0.00 26.91 31.43 53.90 50.02 351 68.255 2 12.694 80.949
24.39 0.00 83.92 55.31 50.20 360 69.543 2 12.694 82.237
69.32 0.00 34.89 61.22 50.39 400 73.321 2 12.694 86.014
0.00 99.89 31.63 64.99 50.49 411 76.177 2 12.694 88.871
14.92 0.00 95.15 66.85 50.60 420 77.435 2 12.694 90.128
12.86 0.00 48.63 69.41 50.68 436 79.858 2 12.694 92.551
10.72 0.00 74.93 72.08 50.84 472 82.262 2 12.694 94.956
0.00 99.99 40.78 84.72 51.10 525 87.315 2 12.694 100.009
84.05 0.00 20.84 87.99 51.36 576 91.270 2 12.694 103.964
70.81 0.00 69.31 98.44 51.71 654 95.706 2 12.694 108.399
49.77 0.00 87.15 54.65 50.76 736 100.260 3 19.939 120.199
0.00 99.88 12.19 61.49 51.00 771 102.669 3 19.939 122.608
73.15 0.00 76.30 63.51 51.13 790 103.972 3 19.939 123.910
0.00 99.40 33.93 64.76 51.20 803 104.998 3 19.939 124.936
0.00 12.99 31.24 67.04 51.26 819 105.884 3 19.939 125.823
0.00 99.82 90.15 70.71 51.45 843 107.486 3 19.939 127.425
54.32 0.00 61.73 75.59 51.61 876 110.524 3 19.939 130.463
99.94 0.00 71.28 84.78 51.83 898 112.780 3 19.939 132.718
0.00 99.83 55.19 88.67 51.87 905 114.009 3 19.939 133.948
0.00 99.81 52.61 90.23 51.92 917 115.439 3 19.939 135.378
58.25 0.00 43.43 92.23 51.98 946 117.746 3 19.939 137.685
0.00 48.37 6.90 97.14 52.01 957 121.232 3 19.939 141.171
0.00 99.92 10.04 97.83 52.08 961 122.171 3 19.939 142.110
73.00 0.00 26.08 98.85 52.16 966 122.969 3 19.939 142.907
67.50 0.00 61.91 40.93 50.85 976 124.105 4 28.736 152.841
0.00 99.81 85.40 42.82 50.92 989 125.620 4 28.736 154.356
84.41 0.00 17.19 45.82 51.22 1042 129.125 4 28.736 157.861
0.00 99.50 49.82 59.85 51.45 1091 133.638 4 28.736 162.375
0.00 55.21 94.22 69.16 51.69 1183 138.147 4 28.736 166.883
90.92 0.00 93.80 82.97 51.78 1198 149.152 4 28.736 177.888
0.00 99.84 74.55 87.53 51.93 1235 152.321 4 28.736 181.057
0.00 29.59 86.43 92.35 52.05 1259 158.220 4 28.736 186.956
0.00 99.88 37.43 94.33 52.13 1265 159.567 4 28.736 188.304
71.51 0.00 92.98 95.85 52.23 1274 160.922 4 28.736 189.658
0.00 58.98 35.24 44.45 53.30 1293 164.024 5 44.472 208.496
72.93 0.00 87.13 48.08 53.55 1306 167.148 5 44.472 211.620
60.62 0.00 16.32 48.79 53.68 1312 168.038 5 44.472 212.510
0.00 99.93 36.39 50.14 53.78 1321 169.574 5 44.472 214.046
57.22 0.00 52.20 50.73 54.02 1326 170.086 5 44.472 214.558
46.74 0.00 22.84 52.39 54.10 1336 171.200 5 44.472 215.672
88.34 0.00 95.61 54.54 54.24 1354 173.729 5 44.472 218.202
0.00 99.56 71.25 55.78 54.31 1361 174.546 5 44.472 219.018
19.66 0.00 78.90 55.97 54.40 1368 174.994 5 44.472 219.466
S0 S1 E O P YGC YGCT FGC FGCT GCT
0.00 99.94 79.71 60.76 97.11 45 6.620 0 0.000 6.620
0.00 82.19 82.81 76.25 97.77 73 15.281 0 0.000 15.281
10.47 0.00 52.29 84.19 98.88 86 18.600 0 0.000 18.600
31.04 0.00 36.31 88.70 99.29 106 21.773 0 0.000 21.773
26.61 0.00 66.35 94.90 99.82 128 26.492 0 0.000 26.492
0.00 99.96 42.14 46.33 49.24 149 32.820 1 7.294 40.114
33.62 0.00 39.03 48.52 49.47 164 35.088 1 7.294 42.381
0.00 86.70 95.99 51.93 49.64 183 38.241 1 7.294 45.534
0.00 99.99 28.78 62.55 49.90 203 43.577 1 7.294 50.871
37.49 0.00 69.00 65.38 50.04 216 46.585 1 7.294 53.879
0.00 99.76 61.33 72.21 50.27 239 49.508 1 7.294 56.802
100.00 0.00 43.33 82.31 50.44 268 51.944 1 7.294 59.238
0.00 72.73 34.29 98.09 50.55 297 58.759 1 7.294 66.053
34.75 0.00 51.93 36.49 44.42 312 61.483 2 12.694 74.176
0.00 99.97 95.84 44.46 49.80 323 63.397 2 12.694 76.091
0.00 26.91 31.43 53.90 50.02 351 68.255 2 12.694 80.949
24.39 0.00 83.92 55.31 50.20 360 69.543 2 12.694 82.237
69.32 0.00 34.89 61.22 50.39 400 73.321 2 12.694 86.014
0.00 99.89 31.63 64.99 50.49 411 76.177 2 12.694 88.871
14.92 0.00 95.15 66.85 50.60 420 77.435 2 12.694 90.128
12.86 0.00 48.63 69.41 50.68 436 79.858 2 12.694 92.551
10.72 0.00 74.93 72.08 50.84 472 82.262 2 12.694 94.956
0.00 99.99 40.78 84.72 51.10 525 87.315 2 12.694 100.009
84.05 0.00 20.84 87.99 51.36 576 91.270 2 12.694 103.964
70.81 0.00 69.31 98.44 51.71 654 95.706 2 12.694 108.399
49.77 0.00 87.15 54.65 50.76 736 100.260 3 19.939 120.199
0.00 99.88 12.19 61.49 51.00 771 102.669 3 19.939 122.608
73.15 0.00 76.30 63.51 51.13 790 103.972 3 19.939 123.910
0.00 99.40 33.93 64.76 51.20 803 104.998 3 19.939 124.936
0.00 12.99 31.24 67.04 51.26 819 105.884 3 19.939 125.823
0.00 99.82 90.15 70.71 51.45 843 107.486 3 19.939 127.425
54.32 0.00 61.73 75.59 51.61 876 110.524 3 19.939 130.463
99.94 0.00 71.28 84.78 51.83 898 112.780 3 19.939 132.718
0.00 99.83 55.19 88.67 51.87 905 114.009 3 19.939 133.948
0.00 99.81 52.61 90.23 51.92 917 115.439 3 19.939 135.378
58.25 0.00 43.43 92.23 51.98 946 117.746 3 19.939 137.685
0.00 48.37 6.90 97.14 52.01 957 121.232 3 19.939 141.171
0.00 99.92 10.04 97.83 52.08 961 122.171 3 19.939 142.110
73.00 0.00 26.08 98.85 52.16 966 122.969 3 19.939 142.907
67.50 0.00 61.91 40.93 50.85 976 124.105 4 28.736 152.841
0.00 99.81 85.40 42.82 50.92 989 125.620 4 28.736 154.356
84.41 0.00 17.19 45.82 51.22 1042 129.125 4 28.736 157.861
0.00 99.50 49.82 59.85 51.45 1091 133.638 4 28.736 162.375
0.00 55.21 94.22 69.16 51.69 1183 138.147 4 28.736 166.883
90.92 0.00 93.80 82.97 51.78 1198 149.152 4 28.736 177.888
0.00 99.84 74.55 87.53 51.93 1235 152.321 4 28.736 181.057
0.00 29.59 86.43 92.35 52.05 1259 158.220 4 28.736 186.956
0.00 99.88 37.43 94.33 52.13 1265 159.567 4 28.736 188.304
71.51 0.00 92.98 95.85 52.23 1274 160.922 4 28.736 189.658
0.00 58.98 35.24 44.45 53.30 1293 164.024 5 44.472 208.496
72.93 0.00 87.13 48.08 53.55 1306 167.148 5 44.472 211.620
60.62 0.00 16.32 48.79 53.68 1312 168.038 5 44.472 212.510
0.00 99.93 36.39 50.14 53.78 1321 169.574 5 44.472 214.046
57.22 0.00 52.20 50.73 54.02 1326 170.086 5 44.472 214.558
46.74 0.00 22.84 52.39 54.10 1336 171.200 5 44.472 215.672
88.34 0.00 95.61 54.54 54.24 1354 173.729 5 44.472 218.202
0.00 99.56 71.25 55.78 54.31 1361 174.546 5 44.472 219.018
19.66 0.00 78.90 55.97 54.40 1368 174.994 5 44.472 219.466
Thank you.
Manish
October 8th, 2011 on 1:53 pm
“I was monitoring the Heap Size and i observed that the Heap is growing to 100% and after Full GC it is settling down to 35-40% , could you please explain this behavior”
This is the way for the JVM to optimize for throughput, i.e., giving the Java threads as much processing time as possible. In this case minor collections occur frequenlty, until the memory reaches a certain limit, usually about 95% it starts planning for a full collection. A nice illustration of this behavior is depicted here: http://middlewaremagic.com/weblogic/wp-content/uploads/2011/07/clientromemoryoverview.png
“Is it normal that the Heap settling down to 40% of its capacity?”
This depends on the number of live objects in your heap, the garbage collector only collects ‘dead’ object (objects that are not referenced any longer.
Some background information can be read here: http://middlewaremagic.com/weblogic/?p=6930 (especially the memory management section)
October 10th, 2011 on 8:25 pm
Thanks Rene. That was helpful.
Manish
November 7th, 2011 on 10:28 am
Hi Rene / Ravish / Jay,
Can someone help me understand , the order of “loading” while we start our weblogic server ?
For example – my weblogic domain has been configured to have :
1. 3 Data sources
2. JMS resources ( Queues and topics )
3. Deployed an web application (active)
I just want to understand and know the order like which will be picked up first , and so before the server come Up.
Thanks
kiran
November 7th, 2011 on 9:51 pm
Hi Kiran,
Usually WebLogic Server deploys server-level resources (first JDBC and then JMS) before deploying application modules. Modules are then deployed in order starting with connector modules, followed by EJB modules, and finally Web Applications.
The actual deployment order of modules is determined by their Load Order attribute. By default, new applications and modules are configured with a Load Order value of 100. Modules with a lower Load Order value are deployed before those with a higher value during startup. Modules with the same Load Order value are deployed in alphabetical order using the deployment name.
You can change the deployment order for modules from the admin console. The actual deployment order of modules is determined by their Deployment Order attribute.
NOTE: But in many times it is required that we need to set the deployment order for different modules on our own … if we see any issue.
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
November 8th, 2011 on 11:39 am
Thanks Jay
This makes me clear about “Load Order” concept.
Thanks
Kiran
November 15th, 2011 on 11:55 am
Hi,
I’m investigating an issue that occurred in our weblogic in production. The weblogic console hangs and the cpu utilization is high. Can you help me analyze what possible cause of this? Here’s thread dump:
“Main Thread” prio=5 id=0x80 pid=11252 waiting
— Waiting for notification on: weblogic/t3/srvr/T3Srvr@0x20eef878[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/t3/srvr/T3Srvr.waitForDeath(T3Srvr.java:1207)@0xb3805e31
^– Lock released while waiting: weblogic/t3/srvr/T3Srvr@0x20eef878[fat lock]
at weblogic/t3/srvr/T3Srvr.run(T3Srvr.java:389)@0xb73bac31
at weblogic/Server.main(Server.java:32)@0xb73ba584
— End of stack trace
“(Signal Handler)” prio=5 id=0x100 pid=11253 active, daemon
“(Code Generation Thread 1)” prio=5 id=0x180 pid=11254 active, native_waiting, daemon
“(Code Optimization Thread 1)” prio=5 id=0x200 pid=11255 active, native_waiting, daemon
“(GC Main Thread)” prio=5 id=0x280 pid=11256 active, daemon
“(GC Worker Thread 1)” prio=5 id=0x300 pid=11257 active, native_waiting, daemon
“(GC Worker Thread 2)” prio=5 id=0x380 pid=11258 active, native_waiting, daemon
“(GC Worker Thread 3)” prio=5 id=0x400 pid=11259 active, native_waiting, daemon
“(GC Worker Thread 4)” prio=5 id=0x480 pid=11260 active, native_waiting, daemon
“(GC Worker Thread 5)” prio=5 id=0x500 pid=11261 active, native_waiting, daemon
“(GC Worker Thread 6)” prio=5 id=0x580 pid=11262 active, native_waiting, daemon
“(GC Worker Thread 7)” prio=5 id=0x600 pid=11263 active, native_waiting, daemon
“(GC Worker Thread 8)” prio=5 id=0x680 pid=11264 active, native_waiting, daemon
“Finalizer” prio=8 id=0x700 pid=11265 waiting, daemon
— Waiting for notification on: java/lang/ref/ReferenceQueue$Lock@0x20088650[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at jrockit/vm/Locks.wait(Optimized Method)@0xb223b03d
at java/lang/ref/ReferenceQueue.remove(ReferenceQueue.java:111)@0xb7399611
^– Lock released while waiting: java/lang/ref/ReferenceQueue$Lock@0x20088650[fat lock]
at java/lang/ref/ReferenceQueue.remove(ReferenceQueue.java:127)@0xb739950c
at java/lang/ref/Finalizer$FinalizerThread.run(Unknown Source)@0xb7399464
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“Reference Handler” prio=10 id=0x780 pid=11266 waiting for pending references, daemon
at java/lang/ref/Reference.getPending(Native Method)@0xb739a3e0
at java/lang/ref/Reference.access$000(Unknown Source)@0xb739a455
at java/lang/ref/Reference$ReferenceHandler.run(Unknown Source)@0xb739a360
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“(Hotspot Detector)” prio=10 id=0x800 pid=11267 active, daemon
“Thread-1” prio=5 id=0x880 pid=11272 waiting, daemon
— Waiting for notification on: java/util/TaskQueue@0x21482530[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at java/util/TimerThread.mainLoop(Timer.java:403)@0xb68b6ff0
^– Lock released while waiting: java/util/TaskQueue@0x21482530[fat lock]
at java/util/TimerThread.run(Timer.java:382)@0xb68b6f12
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x900 pid=11273 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c3feb0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c3feb0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘1’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x980 pid=11274 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c405e0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c405e0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘2’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xa00 pid=11275 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c40b28[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c40b28[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘3’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xa80 pid=11276 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c41070[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c41070[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘4’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xb00 pid=11277 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c415b8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c415b8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘5’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xb80 pid=11278 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c41b00[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c41b00[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘6’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xc00 pid=11279 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c42048[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c42048[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘7’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xc80 pid=11280 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c42590[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c42590[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘8’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xd00 pid=11281 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c42ad8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c42ad8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘9’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xd80 pid=11282 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c43020[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c43020[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’10’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xe00 pid=11283 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c43568[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c43568[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’11’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xe80 pid=11284 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c43ab0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c43ab0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’12’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xf00 pid=11285 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c43ff8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c43ff8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’13’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0xf80 pid=11286 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c44540[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c44540[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’14’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1000 pid=11287 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c44a88[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c44a88[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’15’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1080 pid=11288 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c44fd0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c44fd0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’16’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1100 pid=11289 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c45518[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c45518[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’17’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1180 pid=11290 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c45af0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c45af0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’18’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1200 pid=11291 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c46038[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c46038[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’19’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1280 pid=11292 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c46580[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c46580[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’20’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1300 pid=11293 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c46ac8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c46ac8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’21’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1380 pid=11294 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c47010[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c47010[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’22’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1400 pid=11295 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c47558[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c47558[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’23’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1480 pid=11296 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c47aa0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c47aa0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ’24’ for queue: ‘weblogic.kernel.Default'” prio=5 id=0x1500 pid=11297 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c47fa0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c47fa0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘weblogic.kernel.System'” prio=5 id=0x1580 pid=11298 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c49208[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c49208[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘1’ for queue: ‘weblogic.kernel.System'” prio=5 id=0x1600 pid=11299 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c49740[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c49740[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘2’ for queue: ‘weblogic.kernel.System'” prio=5 id=0x1680 pid=11300 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c49c78[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c49c78[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘3’ for queue: ‘weblogic.kernel.System'” prio=5 id=0x1700 pid=11301 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c4a1b0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c4a1b0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘4’ for queue: ‘weblogic.kernel.System'” prio=5 id=0x1780 pid=11302 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23c4a6e8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23c4a6e8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“weblogic.time.TimeEventGenerator” prio=9 id=0x1800 pid=11303 waiting, daemon
— Waiting for notification on: weblogic/time/common/internal/TimeTable@0x23c4b090[fat lock]
at jrockit/vm/Threads.waitForSignalWithTimeout(Native Method)@0xb738e190
at jrockit/vm/Locks.wait(Optimized Method)@0xb223b454
at weblogic/time/common/internal/TimeTable.snooze(Optimized Method)@0xb2290e84
^– Lock released while waiting: weblogic/time/common/internal/TimeTable@0x23c4b090[fat lock]
at weblogic/time/common/internal/TimeEventGenerator.run(TimeEventGenerator.java:118)@0xb5cfb184
at java/lang/Thread.run(Unknown Source)@0xb690a62f
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“weblogic.security.SpinnerRandomSource” prio=5 id=0x1880 pid=11304 waiting, daemon
— Waiting for notification on: java/lang/Object@0x2329ffc8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/security/SpinnerRandomBitsSource.run(SpinnerRandomBitsSource.java:60)@0xb5d01533
^– Lock released while waiting: java/lang/Object@0x2329ffc8[fat lock]
at java/lang/Thread.run(Unknown Source)@0xb690a62f
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘weblogic.admin.HTTP'” prio=5 id=0x1900 pid=11305 active, daemon
at java/util/HashMap.get(Optimized Method)@0xb69471be
at weblogic/management/console/info/AttributeRegistry.getAttribute(AttributeRegistry.java:85)@0xb30d5fe6
at weblogic/management/console/info/Attribute$Factory.getInstance(Attribute.java:308)@0xb30d5a8a
at weblogic/management/console/info/CompositeAttribute.(CompositeAttribute.java:58)@0xb30dc9dd
at weblogic/management/console/info/Attribute$Factory.getInstance(Attribute.java:314)@0xb30d5f36
at weblogic/management/console/tags/form/ControlTag.setAttribute(ControlTag.java:77)@0xb30d58cc
at weblogic/management/console/webapp/_domain/__machine._jspService(__machine.java:520)@0xb26fa684
at weblogic/servlet/jsp/JspBase.service(JspBase.java:33)@0xb384257b
at weblogic/servlet/internal/ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1072)@0xb3842525
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:465)@0xb3841b24
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:348)@0xb3841868
at weblogic/servlet/internal/RequestDispatcherImpl.forward(RequestDispatcherImpl.java:322)@0xb3840c04
at weblogic/servlet/jsp/PageContextImpl.forward(PageContextImpl.java:150)@0xb386118a
at weblogic/management/console/actions/ForwardAction.perform(ForwardAction.java:35)@0xb3861126
at weblogic/management/console/actions/internal/ActionServlet.doAction(ActionServlet.java:182)@0xb385b10b
at weblogic/management/console/actions/internal/ActionServlet.doGet(ActionServlet.java:92)@0xb385ac10
at javax/servlet/http/HttpServlet.service(HttpServlet.java:740)@0xb385ab25
at javax/servlet/http/HttpServlet.service(HttpServlet.java:853)@0xb385a89e
at weblogic/servlet/internal/ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1072)@0xb3842525
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:465)@0xb3841b24
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:348)@0xb3841868
at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6981)@0xb385a768
at weblogic/security/acl/internal/AuthenticatedSubject.doAs(Optimized Method)@0xb2f1ef3b
at weblogic/security/service/SecurityManager.runAs(Optimized Method)@0xb3882c27
at weblogic/servlet/internal/WebAppServletContext.invokeServlet(WebAppServletContext.java:3892)@0xb3819d24
at weblogic/servlet/internal/ServletRequestImpl.execute(ServletRequestImpl.java:2766)@0xb381883e
at weblogic/kernel/ExecuteThread.execute(Optimized Method)@0xb3895a70
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:183)@0xb5cfa8ce
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘1’ for queue: ‘weblogic.admin.HTTP'” prio=5 id=0x1980 pid=11306 active, daemon
at java/util/HashMap.get(Optimized Method)@0xb69471be
at weblogic/management/console/info/AttributeRegistry.getAttribute(Optimized Method)@0xb2284521
at weblogic/management/console/info/Attribute$Factory.getInstance(Optimized Method)@0xb2284561
at weblogic/management/console/info/CompositeAttribute.(CompositeAttribute.java:58)@0xb30dc9dd
at weblogic/management/console/info/Attribute$Factory.getInstance(Optimized Method)@0xb2284b08
at weblogic/management/console/tags/form/ControlTag.setAttribute(ControlTag.java:77)@0xb30d58cc
at weblogic/management/console/webapp/_domain/__machine._jspService(__machine.java:520)@0xb26fa684
at weblogic/servlet/jsp/JspBase.service(JspBase.java:33)@0xb384257b
at weblogic/servlet/internal/ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1072)@0xb3842525
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:465)@0xb3841b24
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:348)@0xb3841868
at weblogic/servlet/internal/RequestDispatcherImpl.forward(RequestDispatcherImpl.java:322)@0xb3840c04
at weblogic/servlet/jsp/PageContextImpl.forward(PageContextImpl.java:150)@0xb386118a
at weblogic/management/console/actions/ForwardAction.perform(ForwardAction.java:35)@0xb3861126
at weblogic/management/console/actions/internal/ActionServlet.doAction(ActionServlet.java:182)@0xb385b10b
at weblogic/management/console/actions/internal/ActionServlet.doGet(ActionServlet.java:92)@0xb385ac10
at javax/servlet/http/HttpServlet.service(HttpServlet.java:740)@0xb385ab25
at javax/servlet/http/HttpServlet.service(HttpServlet.java:853)@0xb385a89e
at weblogic/servlet/internal/ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1072)@0xb3842525
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:465)@0xb3841b24
at weblogic/servlet/internal/ServletStubImpl.invokeServlet(ServletStubImpl.java:348)@0xb3841868
at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:6981)@0xb385a768
at weblogic/security/acl/internal/AuthenticatedSubject.doAs(Optimized Method)@0xb2f1ef3b
at weblogic/security/service/SecurityManager.runAs(Optimized Method)@0xb3882c27
at weblogic/servlet/internal/WebAppServletContext.invokeServlet(WebAppServletContext.java:3892)@0xb3819d24
at weblogic/servlet/internal/ServletRequestImpl.execute(ServletRequestImpl.java:2766)@0xb381883e
at weblogic/kernel/ExecuteThread.execute(Optimized Method)@0xb3895a70
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:183)@0xb5cfa8ce
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘weblogic.admin.RMI'” prio=5 id=0x1a00 pid=11307 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23ca8fe0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23ca8fe0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘1’ for queue: ‘weblogic.admin.RMI'” prio=5 id=0x1a80 pid=11308 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23ca9528[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23ca9528[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘2’ for queue: ‘weblogic.admin.RMI'” prio=5 id=0x1b00 pid=11309 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x23ca9a50[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x23ca9a50[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“VDE Transaction Processor Thread” prio=2 id=0x1b80 pid=11310 waiting
— Waiting for notification on: com/octetstring/vde/backend/standard/TransactionProcessor@0x23265b00[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at com/octetstring/vde/backend/standard/TransactionProcessor.waitChange(TransactionProcessor.java:365)@0xb5d16455
^– Lock released while waiting: com/octetstring/vde/backend/standard/TransactionProcessor@0x23265b00[fat lock]
at com/octetstring/vde/backend/standard/TransactionProcessor.run(TransactionProcessor.java:212)@0xb5d155bd
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“VDE Replication Thread” prio=5 id=0x1c00 pid=11311 waiting
— Waiting for notification on: com/octetstring/vde/replication/Replication@0x23ce4008[fat lock]
at jrockit/vm/Threads.waitForSignalWithTimeout(Native Method)@0xb738e190
at jrockit/vm/Locks.wait(Optimized Method)@0xb223b454
at com/octetstring/vde/replication/Replication.wait30sec(Replication.java:395)@0xb5d1cabe
^– Lock released while waiting: com/octetstring/vde/replication/Replication@0x23ce4008[fat lock]
at com/octetstring/vde/replication/Replication.run(Replication.java:365)@0xb5d1c97d
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“Thread-5” prio=5 id=0x1c80 pid=11313 waiting
— Waiting for notification on: java/util/TaskQueue@0x25a060e0[fat lock]
at jrockit/vm/Threads.waitForSignalWithTimeout(Native Method)@0xb738e190
at jrockit/vm/Locks.wait(Optimized Method)@0xb223b454
at java/util/TimerThread.mainLoop(Timer.java:429)@0xb68b7129
^– Lock released while waiting: java/util/TaskQueue@0x25a060e0[fat lock]
at java/util/TimerThread.run(Timer.java:382)@0xb68b6f12
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“[JRockit] GCEventThread” prio=5 id=0x1d00 pid=11314 active, native_waiting, daemon
at jrockit/management/jvm/GarbageCollectorImpl.waitForEvent(Native Method)@0xb4c19840
at jrockit/management/jvm/GarbageCollectorImpl.access$000(Unknown Source)@0xb4c198b5
at jrockit/management/jvm/GarbageCollectorImpl$GCEventThread.getNextEvent(Unknown Source)@0xb4c1978e
at jrockit/management/jvm/AsyncEventProvider$EventThread.run(Unknown Source)@0xb4c196d2
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“weblogic.health.CoreHealthMonitor” prio=5 id=0x1d80 pid=11315 waiting, daemon
— Waiting for notification on: java/lang/Object@0x2642dd10[fat lock]
at jrockit/vm/Threads.waitForSignalWithTimeout(Native Method)@0xb738e190
at jrockit/vm/Locks.wait(Optimized Method)@0xb223b454
at java/lang/Thread.sleep(Unknown Source)@0xb690a3d5
^– Lock released while waiting: java/lang/Object@0x2642dd10[fat lock]
at weblogic/t3/srvr/CoreHealthMonitorThread.run(CoreHealthMonitorThread.java:163)@0xb4c49b60
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘JMS.TimerTreePool'” prio=5 id=0x1e00 pid=11316 waiting, daemon
— Waiting for notification on: weblogic/jms/backend/BETimerTree@0x21eb7140[fat lock]
at jrockit/vm/Threads.waitForSignalWithTimeout(Native Method)@0xb738e190
at weblogic/jms/backend/BETimerTree.execute(Optimized Method)@0xb3894e8d
^– Lock released while waiting: weblogic/jms/backend/BETimerTree@0x21eb7140[fat lock]
at weblogic/kernel/ExecuteThread.execute(Optimized Method)@0xb3895a70
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:183)@0xb5cfa8ce
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“Thread-6” prio=5 id=0x1e80 pid=11317 waiting, daemon
— Waiting for notification on: java/util/TaskQueue@0x21f76f70[fat lock]
at jrockit/vm/Threads.waitForSignalWithTimeout(Native Method)@0xb738e190
at jrockit/vm/Locks.wait(Optimized Method)@0xb223b454
at java/util/TimerThread.mainLoop(Timer.java:429)@0xb68b7129
^– Lock released while waiting: java/util/TaskQueue@0x21f76f70[fat lock]
at java/util/TimerThread.run(Timer.java:382)@0xb68b6f12
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ListenThread.Default” prio=5 id=0x1f00 pid=11318 active
at jrockit/net/SocketNativeIO.accept(Native Method)@0xb37ebb70
at jrockit/net/SocketNativeIO.accept(Unknown Source)@0xb37ebc3b
at java/net/AbstractSocketImpl.accept(Unknown Source)@0xb37eba6b
^– Holding lock: java/net/PlainSocketImpl@0x23288a70[thin lock]
at java/net/ServerSocket.implAccept(ServerSocket.java:448)@0xb37eb973
at java/net/ServerSocket.accept(ServerSocket.java:419)@0xb37eb86d
at weblogic/socket/WeblogicServerSocket.accept(WeblogicServerSocket.java:26)@0xb38031fa
at weblogic/t3/srvr/ListenThread.accept(ListenThread.java:735)@0xb38031d7
at weblogic/t3/srvr/ListenThread.run(ListenThread.java:301)@0xb37de904
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“SSLListenThread.Default” prio=5 id=0x1f80 pid=11319 active
at jrockit/net/SocketNativeIO.accept(Native Method)@0xb37ebb70
at jrockit/net/SocketNativeIO.accept(Unknown Source)@0xb37ebc3b
at java/net/AbstractSocketImpl.accept(Unknown Source)@0xb37eba6b
^– Holding lock: java/net/PlainSocketImpl@0x232aad80[thin lock]
at java/net/ServerSocket.implAccept(ServerSocket.java:448)@0xb37eb973
at javax/net/ssl/impl/SSLServerSocketImpl.accept(Unknown Source)@0xb38032b4
at weblogic/t3/srvr/ListenThread.accept(ListenThread.java:735)@0xb38031d7
at weblogic/t3/srvr/ListenThread.run(ListenThread.java:301)@0xb37de904
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘weblogic.socket.Muxer'” prio=5 id=0x2000 pid=11320 active, daemon
at weblogic/socket/PosixSocketMuxer.poll(Native Method)@0xb3810b40
at weblogic/socket/PosixSocketMuxer.processSockets(PosixSocketMuxer.java:100)@0xb38003fc
^– Holding lock: java/lang/String@0x23282900[fat lock]
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:32)@0xb380033a
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:224)@0xb5dbfa10
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:183)@0xb5cfa8ce
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” prio=5 id=0x2080 pid=11321 blocked, daemon
— Blocked trying to get lock: java/lang/String@0x23282900[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.lockFat(Optimized Method)@0xb312a2e9
at jrockit/vm/Locks.monitorEnterSecondStage(Optimized Method)@0xb3126e44
at jrockit/vm/Locks.monitorEnter(Native Method)@0xb738b694
at weblogic/socket/PosixSocketMuxer.processSockets(PosixSocketMuxer.java:91)@0xb380039c
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:32)@0xb380033a
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:224)@0xb5dbfa10
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:183)@0xb5cfa8ce
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘2’ for queue: ‘weblogic.socket.Muxer'” prio=5 id=0x2100 pid=11322 blocked, daemon
— Blocked trying to get lock: java/lang/String@0x23282900[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.fatLockWaitOrSpin(Optimized Method)@0xb6955321
at jrockit/vm/Locks.fatLockReacquire(Optimized Method)@0xb223f5f1
at jrockit/vm/Locks.lockFat(Optimized Method)@0xb312a2f9
at jrockit/vm/Locks.monitorEnterSecondStage(Optimized Method)@0xb3126e44
at jrockit/vm/Locks.monitorEnter(Native Method)@0xb738b694
at weblogic/socket/PosixSocketMuxer.processSockets(PosixSocketMuxer.java:91)@0xb380039c
at weblogic/socket/SocketReaderRequest.execute(SocketReaderRequest.java:32)@0xb380033a
at weblogic/kernel/ExecuteThread.execute(ExecuteThread.java:224)@0xb5dbfa10
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:183)@0xb5cfa8ce
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘0’ for queue: ‘JMS.TimerClientPool'” prio=5 id=0x2180 pid=11324 waiting, daemon
— Blocked trying to get lock: weblogic/kernel/ExecuteThreadManager@0x214ba060[thin lock]
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x21e9acf0[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x21e9acf0[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘1’ for queue: ‘JMS.TimerClientPool'” prio=5 id=0x2200 pid=11325 waiting, daemon
— Blocked trying to get lock: weblogic/kernel/ExecuteThreadManager@0x214ba060[thin lock]
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x2325c688[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x2325c688[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘2’ for queue: ‘JMS.TimerClientPool'” prio=5 id=0x2280 pid=11326 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x2325cc18[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x2325cc18[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘3’ for queue: ‘JMS.TimerClientPool'” prio=5 id=0x2300 pid=11327 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x2325d1a8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x2325d1a8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘4’ for queue: ‘JMS.TimerClientPool'” prio=5 id=0x2380 pid=11328 waiting, daemon
— Blocked trying to get lock: weblogic/kernel/ExecuteThreadManager@0x214ba060[thin lock]
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x2325d738[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Unknown Source)@0xb7399a24
at weblogic/kernel/ExecuteThread.waitForRequest(ExecuteThread.java:154)@0xb5cfab0b
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x2325d738[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“ExecuteThread: ‘5’ for queue: ‘JMS.TimerClientPool'” prio=5 id=0x2400 pid=11329 waiting, daemon
— Waiting for notification on: weblogic/kernel/ExecuteThread@0x2325dcf8[fat lock]
at jrockit/vm/Threads.waitForSignal(Native Method)@0xb73997c0
at jrockit/vm/Locks.wait(Optimized Method)@0xb3130f2a
at weblogic/kernel/ExecuteThread.waitForRequest(Optimized Method)@0xb221ef66
^– Lock released while waiting: weblogic/kernel/ExecuteThread@0x2325dcf8[fat lock]
at weblogic/kernel/ExecuteThread.run(ExecuteThread.java:174)@0xb5cfa8b9
at java/lang/Thread.startThreadFromVM(Unknown Source)@0xb73993b3
— End of stack trace
“Thread-16” prio=5 id=0x2480 pid=23439 active, daemon
at jrockit/net/SocketNativeIO.read(Native Method)@0xb37ed2f0
at jrockit/net/SocketNativeIO.read(Unknown Source)@0xb37ed361
at java/net/AbstractSocketImpl$1.read(Optimized Method)@0xb312bf11
at jrockit/io/NativeIOInputStream.read(Optimized Method)@0xb2242179
at java/io/BufferedInputStream.fill(BufferedInputStream.java:183)@0xb68b98fd
at java/io/BufferedInputStream.read1(BufferedInputStream.java:222)@0xb73bebf5
at java/io/BufferedInputStream.read(BufferedInputStream.java:277)@0xb73beadb
^– Holding lock: java/io/BufferedInputStream@0x23d2ad80[thin lock]
at com/sun/jndi/ldap/Connection.run(Connection.java:780)@0xb2ef7d0c
at java/lang/Thread.run(Unknown Source)@0xb690a62f
at java/lang/Thread.startThreadFromVM(Optimized Method)@0xb312a12a
— End of stack trace
Blocked lock chains
===================
Chain 2:
“ExecuteThread: ‘2’ for queue: ‘weblogic.socket.Muxer'” (id: 0x00002100) waiting for java/lang/String@0x23282900 held by:
“ExecuteThread: ‘0’ for queue: ‘weblogic.socket.Muxer'” (id: 0x00002000) in chain 1
Open lock chains
================
Chain 1:
“ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” (id: 0x00002080) waiting for java/lang/String@0x23282900 held by:
“ExecuteThread: ‘0’ for queue: ‘weblogic.socket.Muxer'” (id: 0x00002000) (active)
===== END OF THREAD DUMP ===============
THANK YOU!!
November 15th, 2011 on 6:57 pm
Hi Eina,
High CPU usually causes Server hang, So we will have to first find out Which Thread is consuming High CPU. just by looking at the thread dump it is not possible to find out which Thread is consuming High CPU so we require the “Thread Dump” as well as the “top” command results to map the results.
You can refer to the following article to know what all things are required in order to investigate High CPU issues: http://middlewaremagic.com/weblogic/?p=4884
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
December 28th, 2011 on 2:49 pm
Hi jay
I am facing a problem, java.net.SocketException: Socket Closed
The Weblogic server showing up and running but the service is not available.Due to this i am doing bouncing every time . need your help here.please find the below exception
>
#### <> 11/12/27 21:44:32:947: (ter.ets.HttpChan
nel.doPost) [Warning] getReturnMapFromException returned null, ex = com.xxxx.ets.exception.EtsConfigurationException: Return Map not found for Name:
GeneralRpaFailure
>
#### 11/12/27 21:44:32:939: (er.rpa.RpaManager
.dispatch) [Warning] getReturnMapFromException returned null, ex = java.rmi.RemoteException: SOAP Fault:javax.xml.rpc.soap.SOAPFaultException: Failed to sen
d request
Detail:
java.net.SocketException: Socket Closed
at java.net.PlainSocketImpl.setOption(PlainSocketImpl.java:177)
at java.net.Socket.setTcpNoDelay(Socket.java:771)
at weblogic.net.http.HttpClient.openServer(HttpClient.java:305)
at weblogic.net.http.HttpClient.openServer(HttpClient.java:354)
at weblogic.net.http.HttpClient.<init>(HttpClient.java:139)
at weblogic.net.http.HttpClient.New(HttpClient.java:212)
at weblogic.net.http.HttpURLConnection.connect(HttpURLConnection.java:140)
at weblogic.net.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:189)
at weblogic.webservice.binding.http11.Http11ClientBinding.send(Http11ClientBinding.java:427)
at weblogic.webservice.core.handler.ClientHandler.handleRequest(ClientHandler.java:37)
at weblogic.webservice.core.HandlerChainImpl.handleRequest(HandlerChainImpl.java:143)
at weblogic.webservice.core.ClientDispatcher.send(ClientDispatcher.java:231)
at weblogic.webservice.core.ClientDispatcher.dispatch(ClientDispatcher.java:143)
at weblogic.webservice.core.DefaultOperation.invoke(DefaultOperation.java:471)
at weblogic.webservice.core.DefaultOperation.invoke(DefaultOperation.java:457)
at weblogic.webservice.core.rpc.StubImpl._invoke(StubImpl.java:303)
at com.prairiesys.IVBXML.RPA_PortType_Stub.getAddress(RPA_PortType_Stub.java:26)
at com.xxxx.rpa.prairie.RPAPrairieWSClient.getAddress(Lcom.prairiesys.www.IVBXML.GetAddress;)Lcom.prairiesys.www.IVBXML.GetAddressResponse;(Unk
nown Source)
at com.xxxx.rpa.cpa.CpaDispatcher.getAddress(Lcom.xxxx.rpa.mesg.RpaAdressRequestMessage;Ljava.lang.String;)Lcom.xxxx.rpa.mesg.Rpa
ResponseMessage;(Unknown Source)
at com.r.rpa.RpaManager.dispatch(Lcom.xxxx.ets.mesg.EtsRequestMessage;)Lcom.xxxx.ets.mesg.EtsResponseMessage;(Unknown Source)
at com.xxxx.ets.EtsRouter.dispatch(Lcom.xxxx.ets.mesg.EtsRequestMessage;)Lcom.xxxx.ets.mesg.EtsResponseMessage;(Unknown Source)
at com.xxxx.ets.HttpChannel.doPost(Ljavax.servlet.http.HttpServletRequest;Ljavax.servlet.http.HttpServletResponse;)V(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1077)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:465)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:348)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:7047)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3902)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2773)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
:
; nested exception is:
javax.xml.rpc.soap.SOAPFaultException: Failed to send request
>
#### <>
#### <>
#### <> <A stack trace associated with message 220048 follows:
java.net.SocketException: Socket Closed
at java.net.PlainSocketImpl.setOption(PlainSocketImpl.java:177)
at java.net.Socket.setTcpNoDelay(Socket.java:771)
at weblogic.net.http.HttpClient.openServer(HttpClient.java:305)
at weblogic.net.http.HttpClient.openServer(HttpClient.java:354)
at weblogic.net.http.HttpClient.(HttpClient.java:139)
at weblogic.net.http.HttpClient.New(HttpClient.java:212)
at weblogic.net.http.HttpURLConnection.connect(HttpURLConnection.java:140)
at weblogic.net.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:189)
at weblogic.webservice.binding.http11.Http11ClientBinding.send(Http11ClientBinding.java:427)
at weblogic.webservice.core.handler.ClientHandler.handleRequest(ClientHandler.java:37)
at weblogic.webservice.core.HandlerChainImpl.handleRequest(HandlerChainImpl.java:143)
at weblogic.webservice.core.ClientDispatcher.send(ClientDispatcher.java:231)
at weblogic.webservice.core.ClientDispatcher.dispatch(ClientDispatcher.java:143)
at weblogic.webservice.core.DefaultOperation.invoke(DefaultOperation.java:471)
at weblogic.webservice.core.DefaultOperation.invoke(DefaultOperation.java:457)
at weblogic.webservice.core.rpc.StubImpl._invoke(StubImpl.java:303)
at com.prairiesys.IVBXML.RPA_PortType_Stub.getAddress(RPA_PortType_Stub.java:26)
at com.xxxx.rpa.prairie.RPAPrairieWSClient.getAddress(Lcom.prairiesys.www.IVBXML.GetAddress;)Lcom.prairiesys.www.IVBXML.GetAddressResponse;(Unk
nown Source)
at com.xxxx.rpa.cpa.CpaDispatcher.getAddress(Lcom.xxxx.rpa.mesg.RpaAdressRequestMessage;Ljava.lang.String;)Lcom.xxxx.rpa.mesg.Rpa
ResponseMessage;(Unknown Source)
at com.xxxx.rpa.RpaManager.dispatch(Lcom.xxxx.ets.mesg.EtsRequestMessage;)Lcom.xxxx.ets.mesg.EtsResponseMessage;(Unknown Source)
at com.xxxx.ets.EtsRouter.dispatch(Lcom.xxxx.ets.mesg.EtsRequestMessage;)Lcom.xxxx.ets.mesg.EtsResponseMessage;(Unknown Source)
at com.xxxx.ets.HttpChannel.doPost(Ljavax.servlet.http.HttpServletRequest;Ljavax.servlet.http.HttpServletResponse;)V(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:1077)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:465)
at weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:348)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:7047)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
at weblogic.servlet.internal.WebAppServletContext.invokeServlet(WebAppServletContext.java:3902)
at weblogic.servlet.internal.ServletRequestImpl.execute(ServletRequestImpl.java:2773)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
Please reply ASAP
December 29th, 2011 on 12:26 am
Hi Firoj,
The above exception trace indicates some sort of N./W communication issue. Can you please check whether the service/WSDL address mentioned inside your code ” com.xxxx.rpa.prairie.RPAPrairieWSClient” method getAddress() is reachable or not when you are getting the above exception?
Also If you are using WebLogic 8.1 then enable the following JAVA_OPTION on server as well as on WebService client end to get more detailed informatoins of the WebService invocation failure:
(For WebLogic 8.1 JAVA_OPTION) -Dweblogic.webservice.verbose=true
(For WebLogic 9.x onwards JAVA_OPTION) -Dweblogic.wsee.verbose=*
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
February 21st, 2012 on 9:30 pm
Hi Sir
I need your help here
We are going to migrate the weblogic server from datacenter to another data center.
can you please explain me the basic steps how to proceed
what i need to check in existing environment?
steps in preinstallation of weblogic in target server?
steps after installation ?
could you please explain? I need your valuble info here
thank you
Pathan
April 17th, 2012 on 6:07 am
Hi Team,
Can you please provide me the scripts which automatically sends an email to us when the Weblogic servers are in Warning state or shutdown state. Thank you.
July 5th, 2012 on 11:35 am
Hi,
i am trying to find out the state of n number of message bridges which are configured in my domain. for that i wrote a wlst program. but while execution it failed. Please help me how to find out the state of the message bridges. Find the below wlst script.
connect(‘system’,’password’,’t3://nus690pb:14101′)
servers = domainRuntimeService.getServerRuntimes();
if (len(servers) > 0):
for server in servers:
messagingBridgeRuntimeEntry = server.getMessagingBridgeRuntimeEntry;
messageBridges = messagingBridgeRuntimeEntry.getMessageBridges();
for messageBridge in messageBridges:
destinations = messageBridge.getDestinations();
for destination in destinations:
print ‘ Name ‘ , desitnation.getName()
print ‘ State ‘ , destination.getState()
Please help me.
July 5th, 2012 on 5:02 pm
Looks like the follow is not correct:
server.getMessagingBridgeRuntimeEntry;
You forgot (), i.e.,
server.getMessagingBridgeRuntimeEntry();
If this is not the case it might help if you added the error message as well.
July 6th, 2012 on 2:27 pm
Hi
connect(‘weblogic’,’weblogic’,’t3://localhost:7001′)
servers = domainRuntimeService.getServerRuntimes();
if (len(servers) > 0):
for server in servers:
messagingBridgeRuntimeEntry = server.getMessagingBridgeRuntimeEntry();
messageBridges = messagingBridgeRuntimeEntry.getMessageBridges();
for messageBridge in messageBridges:
destinations = messageBridge.getDestinations();
for destination in destinations:
print ‘ Name ‘ , desitnation.getName()
print ‘ State ‘ , destination.getState()
As you said made the changes to the above script was failed to execute. Find the error log details below and provide me how to get the state of a message bridge using wlst.
C:Documents and Settingssiva_guddetiDesktop>wlst JMSBridgeRuntime.py
CLASSPATH=d:beapatch_weblogic924profilesdefaultsys_manifest_classpathweblo
gic_patch.jar;d:beajrockit_150_22libtools.jar;D:beaweblogic92serverlibw
eblogic_sp.jar;D:beaweblogic92serverlibweblogic.jar;D:beaweblogic92serve
rlibwebservices.jar;
PATH=d:beapatch_weblogic924profilesdefaultnative;D:beaweblogic92servern
ativewin32;D:beaweblogic92serverbin;d:beajrockit_150_22jrebin;d:beaj
rockit_150_22bin;D:oracleproduct10.2.0db_1bin;C:SYSROOTsystem32;C:SYSRO
OT;C:SYSROOTSystem32Wbem;C:Program FilesCASharedComponentsScanEngine;C:P
rogram FilesCASharedComponentsCAUpdate;C:Program FilesCASharedComponents
ThirdParty;C:Program FilesCASharedComponentsSubscriptionLicense;C:Program
FilesCAeTrustITM;C:SYSROOTsystem32WindowsPowerShellv1.0;C:Program Files
Microsoft Application Virtualization Client;C:Program FilesWindows Imaging;D:
beajdk150_22bin;C:Program FilesIBMRationalSDLCClearCasebin;C:Program Fi
lesIBMRationalSDLCcommon;C:Program FilesIBMGSK7bin;C:Program FilesIBMG
SK7lib;C:IBMITMbin;C:IBMITMbindll;C:IBMITMInstallITM;C:IBMITMTMAIT
M6;d:beaweblogic92commonbin;;d:beajdk150_22bin;d:beaweblogic92commonb
in;D:beaweblogic92servernativewin32oci920_8
Your environment has been set.
CLASSPATH=d:beapatch_weblogic924profilesdefaultsys_manifest_classpathweblo
gic_patch.jar;d:beajrockit_150_22libtools.jar;D:beaweblogic92serverlibw
eblogic_sp.jar;D:beaweblogic92serverlibweblogic.jar;D:beaweblogic92serve
rlibwebservices.jar;;D:beaweblogic92commonevalpointbaselibpbembedded51.
jar;D:beaweblogic92commonevalpointbaselibpbtools51.jar;D:beaweblogic92
commonevalpointbaselibpbclient51.jar
Initializing WebLogic Scripting Tool (WLST) …
Welcome to WebLogic Server Administration Scripting Shell
Type help() for help on available commands
Connecting to t3://nus690pb:14101 with userid system …
Successfully connected to Admin Server ‘router_server’ that belongs to domain ‘r
outerdomain’.
Warning: An insecure protocol was used to connect to the
server. To ensure on-the-wire security, the SSL port or
Admin port should be used instead.
Problem invoking WLST – Traceback (innermost last):
File “C:Documents and Settingssiva_guddetiDesktopJMSBridgeRuntime.py”, lin
e 5, in ?
AttributeError: instance of ‘$Proxy5’ has no attribute ‘getMessagingBridgeRuntime’
July 9th, 2012 on 12:43 pm
Note that the ServerRuntimeMBean does not have a relation to the MessagingBridgeRuntimeMBean, so the server.getMessagingBridgeRuntimeEntry() will throw the error you are observing – AttributeError: instance of ‘$Proxy5′ has no attribute ‘getMessagingBridgeRuntime’
See here for a complete MBean reference http://docs.oracle.com/cd/E12839_01/apirefs.1111/e13951/core/index.html
September 25th, 2012 on 7:20 pm
Hi,
I am new to FMW. For single sign on we are using OAM which is configured with Web Server, the app… server remains on another machine. For each environment we are maintaining respective WebTier server( for dev it is having dev mw server which is configured with devwt server, for qa also having the same which is having their own configurations internally). Can I use only one webtier server which is configure with devmw & qamw, so that users can login from single point to multiple environments.
Can you please share some information on this?
Thanks,
Pavan.
Re-using Webtier/OAM for multiple environments that share same AD.
Thanks,
Shravan.
September 26th, 2012 on 6:47 pm
Thanks Rene.
October 15th, 2012 on 6:09 am
Hi,
I am trying to deploy a library that comes with weblogic, but getting the following error. It’s reporting that the library does not have the specification version. It’s not custom build library, I am not sure what’s wrong. pls help me out.
.Default (self-tuning)’> <Console encountered the following error weblogic.application.library.Li
braryDeploymentException: [J2EE:160144]Failed to register library Extension-Name: glassfish.el_2.2.0, Implementation-Version: 2.2.0: Library ca
nnot have Implementation-Version set, without also specifying its Specification-Version.
at weblogic.application.internal.library.LibraryDeployment.prepare(LibraryDeployment.java:49)
at weblogic.application.internal.DeploymentStateChecker.prepare(DeploymentStateChecker.java:154)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.prepare(AppContainerInvoker.java:60)
at weblogic.deploy.internal.targetserver.operations.ActivateOperation.createAndPrepareContainer(ActivateOperation.java:207)
at weblogic.deploy.internal.targetserver.operations.ActivateOperation.doPrepare(ActivateOperation.java:98)
at weblogic.deploy.internal.targetserver.operations.AbstractOperation.prepare(AbstractOperation.java:217)
at weblogic.deploy.internal.targetserver.DeploymentManager.handleDeploymentPrepare(DeploymentManager.java:747)
at weblogic.deploy.internal.targetserver.DeploymentManager.prepareDeploymentList(DeploymentManager.java:1216)
at weblogic.deploy.internal.targetserver.DeploymentManager.handlePrepare(DeploymentManager.java:250)
at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.prepare(DeploymentServiceDispatcher.java:159)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doPrepareCallback(DeploymentReceiverCallbackDelive
rer.java:171)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.access$000(DeploymentReceiverCallbackDeliverer.jav
a:13)
at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer$1.run(DeploymentReceiverCallbackDeliverer.java:46)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
October 16th, 2012 on 2:18 am
Hi Rene,
Thanks for your quick response. No, it does not have the specification version. It does not make any sense to me why weblogic wants the jar file to be edited it’s deployed. Because it’s supplied with Weblogic and it should not expect users to edit it if it is mandatory to have the specification version. Hopefully Oracle would consider including the specification version in the jar file in future. Again, thanks for your help.
—————
Implementation-Title: Expression Language 2.1 for JSP 2.2 Wed Jul 1 01
:34:54 PDT 2009
Implementation-Version: 2.2.0
Source-Repository-Change-Id: 1234052
—————–
October 25th, 2012 on 4:26 am
Hi, I am getting the below error. (note: I replaced the hostname and IPs with dummy values). It appears that he certificate contained the hostname with domain name, but the weblogic check is returning only the hostname with out the domain name. Can you please help me out? I know I can disable the hostname verifier, but I am looking for a solution without disabling the hostname varifier.
————–
Certificate chain received from test1 – 10.00.00.01 failed hostname verification check. Certificate contained test1.raj.com but check expected test1
———
October 25th, 2012 on 7:07 pm
Rene, Thanks for your quick reply. I already disabled the hostname verification and it’s working fine. But as I mentioned in my post, I am looking for a solution without disabling the hostname verification. Do you have any idea on what would be wrong with the weblogic check?
October 25th, 2012 on 11:34 pm
Hi Rene,
Thanks for the info. The hostname verification is failing when I try to start the managed server from the admin console.
October 26th, 2012 on 7:13 pm
Rene, I had a doubt if it was okay to turn it off in the prod. Looks like it should be okay. Thanks for your help.
November 28th, 2012 on 11:43 am
Hi i want to know how to create a domain through silent.xml i am using WebLogic Server 12c in windows xp
thanks in advance
charan
November 28th, 2012 on 5:10 pm
Thanks Rene, It has installation procedure. But I want to create domain by using silent.xml in WLS 12c.
Any help appreciated!!
Thanks,
Charan
November 29th, 2012 on 11:31 am
Thanks Rene
December 7th, 2012 on 11:18 am
Hi,
Recently i face the below issue. once i configured the weblogic resoures using the lock and edit button and saved it. After that when i click the activate button the server going down with the below exception. This happens frequently. i am using the weblogic 9.2 MP3 version. Please help me how to resolve the issue.
12 10:31:33 PM EST>
<There was an error while running a lifecycle stage :: Lifecycle: UIControl.render :: for the control :: null ::.
com.bea.netuix.nf.UIControlException: com.bea.portlet.adapter.scopedcontent.ActionLookupFailedException: java.lang.NoSuchMethodError: org.jdom.Element.addContent(Lorg/jdom/Element;)Lorg/jdom/Element;
at com.bea.netuix.servlets.controls.content.NetuiContent.checkPreRenderExceptions(NetuiContent.java:400)
at com.bea.netuix.servlets.controls.content.NetuiContent.beginRender(NetuiContent.java:339)
at com.bea.netuix.nf.ControlLifecycle$7.visit(ControlLifecycle.java:486)
at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:530)
at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:541)
Truncated. see log file for complete stacktrace
com.bea.portlet.adapter.scopedcontent.ActionLookupFailedException: java.lang.NoSuchMethodError: org.jdom.Element.addContent(Lorg/jdom/Element;)Lorg/jdom/Element;
at com.bea.portlet.adapter.scopedcontent.ScopedContentCommonSupport.executeAction(ScopedContentCommonSupport.java:699)
at com.bea.portlet.adapter.scopedcontent.ScopedContentCommonSupport.processActionInternal(ScopedContentCommonSupport.java:144)
at com.bea.portlet.adapter.scopedcontent.StrutsStubImpl.processAction(StrutsStubImpl.java:76)
at com.bea.portlet.adapter.NetuiActionHandler.raiseScopedAction(NetuiActionHandler.java:99)
at com.bea.netuix.servlets.controls.content.NetuiContent.raiseScopedAction(NetuiContent.java:180)
Truncated. see log file for complete stacktrace
java.lang.NoSuchMethodError: org.jdom.Element.addContent(Lorg/jdom/Element;)Lorg/jdom/Element;
at com.bea.console.preferences.spi.FilePreferencesProvider.writeUserPreferences(FilePreferencesProvider.java:490)
at com.bea.console.preferences.spi.FilePreferencesProvider.writeToFile(FilePreferencesProvider.java:409)
at com.bea.console.preferences.spi.FilePreferencesProvider.store(FilePreferencesProvider.java:221)
at com.bea.portlet.prefs.spi.PreferencePersistenceDelegate.storePreferences(PreferencePersistenceDelegate.java:120)
at com.bea.portlet.prefs.spi.AbstractPortletPreferences.internalStore(AbstractPortletPreferences.java:459)
Truncated. see log file for complete stacktrace
December 11th, 2012 on 7:20 pm
Hi,
Our team has a scenario with BPEL composites migration from one instance to another e.g. from Dev to Test to UAT, etc . When the jar, that we(admins) migrate from one instance to another, has references to first instance (Dev)/previous instance , it creates an issue.
Under this situation, if the first/previous instance is down due to some reason and if the deployed composite in current instance has references to previous instance, the composite doesn’t work as expected.
Practices followed :
Config plans are used during migration from first instance to the next instance , which ‘searches & replaces’ the references to previous instances.
Problem faced :
In spite of strict standard that is being followed, it is human that there are slips and misses. And during such scenarios, when the composites dont work as expected, all hell breaks loose !
Partial Solution :
A partial solution that we found could restrict references to other instances, and could give a clear indication while composite deployment itself, is setting up ‘ Connection Filter’ under Domain > Security > Filter.
This (when configured in first/previous instance) works as access blocking rule i.e. any requests from current instance to previous instance will be blocked ‘at’ previous instance.
The catch here is , when a previous instance is on the same ‘IP’ as the current instance, setting up this connection filter rule doesnt solve the purpose.
Question :
Is there any other method to restrict ‘outgoing requests from current instance’ than blocking incoming requests in previous instances ?
January 26th, 2013 on 2:24 am
Hi,
I am seeing so many .tmp files sitting in /tmp folder and not cleaning up by their own. Can anyone tell me why these files are generating and siting in /tmp and also please let me know how to get rid of these files.
EX: /tmp/1358967927859726047624544989663.tmp
using jrockit_160_29_D1.2.0-10 (weblogic -10.3.6)
Before I used to see .jfr files created in /tmp.. giving -XX:-FlightRecorder in arguments solved that problem, but still seeing.tmp files generating.
Thank you !
April 16th, 2013 on 2:49 pm
Hi Jay,
Can you explain on the below.
1. How to integrate Web Server with Weblogic ?
2. What is the difference between Thread dump / Heap Dump and there purpose ?
Thanks in Advance.
April 18th, 2013 on 1:17 am
Hello All,
I’m wondering if anyone has ever faced the issue where a WLS plugin mod_wl.so is setup on Apache and the Apache instance is hosted in a DMZ where SSL is terminated. The problem comes where I have turned off SSL Listen port on the WLS server which is not in the DMZ and traffic between the Apache WebLogic Plugin and the WebLogic Server in http. SSL is setup on the Apache server and the secure icon is working great until the WLS server uses something like a call to google maps API on https. Now I have mixed mode content and my nice green cert icon gets a red slash. I would prefer not to use the http version of the google API, but that would seemingly fix the cert issue. Does anyone know the real solution? You must be able to terminate SSL at the Plug-in so that traffic can be inspected by an IDS. Maybe I’m missing something. Any comments or suggestions are welcome.
Thanks!
April 18th, 2013 on 1:30 am
I may have posted too soon. The way to do this is likely to be a simple code change to make sure the call to the google API comes back in through the https Apache with WLS Plug-in host and not back to the application on the app Tier. All communication that leaves our network must come back in through the DMZ, it’s that simple.
Just needed to talk (write) this one out.
Thanks! Comments or suggestions are still very much welcome.
April 25th, 2013 on 1:03 am
It seems that my attempt to fix this did not work. Does anyone have advice?
The situation is a N-Tier environment where (WEB – Apache with mod_wl.so proxying to WLS, Application – Weblogic Server, and DB – Oracle instance) SSL is terminated on the Web Tier at Apache and passed on using mod_wl.so to the Application Tier running WebLogic 10.3.3. The Application then makes a call to google API for calculating distance betweeen two points using the https URL. Because I terminate SSL on the Web Tier we now have mixed content and a nasty red x on my https in the URL making users frightened.
The URL the user normally sees is https://server/application where server is the Apache server. How do I fix this? Am I missing something on the WLS setup or is it just fundamental that if the site is http at this point and wants to call https mixed content will occur. There has to be a solution without using google http APIs. Any advice?
May 1st, 2013 on 1:57 am
I have found the solution to the above problem where SSL is terminated at the Web Tier and the Application Tier makes a request to an https URL. You need to get the version of the Apache WLS plugin called 1.1 which does not ship with WLS. Then in your httpd.conf simply set the directive:
WLProxySSL ON
Under certain conditions on of which is descrbed above the directive does the following – When WLProxySSL is set to ON, the location header returned to the client from WebLogic Server specifies the HTTPS protocol.
Hope this helps someone. This website is a great resource! Thank you Magic Team.
July 9th, 2013 on 3:38 pm
Hi Team,
I was looking for some information on Synchronous and Asynchronous data sources in Weblogic. I googled it, but couldn’t get much information. It will be great if you could shed some light on this.
Thank you very much.
Sohel.