93 Comments for this entry

• Animesh Banerjee
  September 26th, 2009 on 7:22 pm

  Wow Jay….
  This is really a good example of using Work Managers..
  This really helps.

  Log in to Reply
  • Sreedevi
    October 21st, 2009 on 10:28 pm

    Hi Joy,

    Very good example. I have so many questions on work managers. Can we discuss on this blog?

    Thanks,
    Sreedevi

    Log in to Reply
    • Joy SenSharma
      December 6th, 2009 on 3:46 am

      Sure …Just Post your Queries here…It will be really very helpful for others as well.

      Log in to Reply
• sometimes
  December 28th, 2009 on 11:21 am

  Hashtable env = new Hashtable();
  WHY USE HashTable instead of HashMap is there any specific reason.

  Log in to Reply
  • Joy SenSharma
    December 28th, 2009 on 12:03 pm

    1). If you are asking Why we cannot use HashMap in the above example:

    Context ctx=new InitialContext(hashMap); //WRONG

    Then it will be wrong because the constructor of javax.naming.InitialContext class doesn’t have any such constructor, whcih takes HashMap as an Argument…So we have to use Hashtable there.

    2). In General sence if you are asking about Why and where we should use java.util.HashMap and where to use java.util.Hashtable?
    Answer:
    (A) java.util.HashMap is not Synchronized class..Means it is not a thread Safe class… Multiple Threads can manipulate the data inside a HashMap object concurrently…So in MultiThreaded Environments java.util.Hashtable is more useful ..because it has many Thread Safe “Synchronized” methods.

    (B) java.util.HashTable cannot accept any Null Key but HashMap can accept one and only one Null Key..
    Example:
    HashMap hmap=new HashMap();
    hmap.put(null,”AAAAAAA”); //CORRECT
    BUT:
    Hashtable ht=new Hashtable();
    ht.put(null,”AAAAAAA”); //WRONG
    So in this case also Hashtable is safe as it restricts us to pass a NULL value as a Key.

    So finally it depends on our requirement ..where to use HashMap and Where to use Hashtable.

    Log in to Reply
  • Joy SenSharma
    January 4th, 2010 on 7:06 am

    1). If you are asking Why we cannot use HashMap in the above example:

    Context ctx=new InitialContext(hashMap); //WRONG

    Then it will be wrong because the constructor of javax.naming.InitialContext class doesn’t have any such constructor, whcih takes HashMap as an Argument…So we have to use Hashtable there.

    2). In General sence if you are asking about Why and where we should use java.util.HashMap and where to use java.util.Hashtable?
    Answer:
    (A) java.util.HashMap is not Synchronized class..Means it is not a thread Safe class… Multiple Threads can manipulate the data inside a HashMap object concurrently…So in MultiThreaded Environments java.util.Hashtable is more useful ..because it has many Thread Safe “Synchronized” methods.

    (B) java.util.HashTable cannot accept any Null Key but HashMap can accept one and only one Null Key..
    Example:
    HashMap hmap=new HashMap();
    hmap.put(null,”AAAAAAA”); //CORRECT
    BUT:
    Hashtable ht=new Hashtable();
    ht.put(null,”AAAAAAA”); //WRONG
    So in this case also Hashtable is safe as it restricts us to pass a NULL value as a Key.

    So finally it depends on our requirement ..where to use HashMap and Where to use Hashtable.

    1). If you are asking Why we cannot use HashMap in the above example:

    Context ctx=new InitialContext(hashMap); //WRONG

    Then it will be wrong because the constructor of javax.naming.InitialContext class doesn’t have any such constructor, whcih takes HashMap as an Argument…So we have to use Hashtable there.

    2). In General sence if you are asking about Why and where we should use java.util.HashMap and where to use java.util.Hashtable?
    Answer:
    (A) java.util.HashMap is not Synchronized class..Means it is not a thread Safe class… Multiple Threads can manipulate the data inside a HashMap object concurrently…So in MultiThreaded Environments java.util.Hashtable is more useful ..because it has many Thread Safe “Synchronized” methods.

    (B) java.util.HashTable cannot accept any Null Key but HashMap can accept one and only one Null Key..
    Example:
    HashMap hmap=new HashMap();
    hmap.put(null,”AAAAAAA”); //CORRECT
    BUT:
    Hashtable ht=new Hashtable();
    ht.put(null,”AAAAAAA”); //WRONG
    So in this case also Hashtable is safe as it restricts us to pass a NULL value as a Key.

    So finally it depends on our requirement ..where to use HashMap and Where to use Hashtable.

    Joy SenSharma

    Log in to Reply
• Jim
  January 24th, 2010 on 8:58 am

  Thanks for replying to my post in oracle forums.

  i’ve created a work manager with constraint thru the admin console. Now how do i link it with the application. (It’s a web app).

  I need to add the work manager tags in the the weblogic.xml file…thats it???

  or do i have to do something else?

  Log in to Reply
  • Joy SenSharma
    January 24th, 2010 on 12:48 pm

    Hi Jim,

    You need to provide the following entry in “weblogic.xml”

    <?xml version=”1.0″ encoding=”UTF-8″?>

    <weblogic-web-app xmlns=”http://www.bea.com/ns/weblogic/90″>

    <wl-dispatch-policy>myConsoleWorkManager</wl-dispatch-policy>

    </weblogic-web-app>

    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
  • Joy SenSharma
    January 24th, 2010 on 12:54 pm

    Hi Jim,

    After providing the following entry in “weblogic.xml”

    <?xml version=”1.0″ encoding=”UTF-8″?>

    <weblogic-web-app xmlns=”http://www.bea.com/ns/weblogic/90″>

    <wl-dispatch-policy>myConsoleWorkManager</wl-dispatch-policy>

    </weblogic-web-app>

    Once you deploy your application….you can check it from admin console as well..

    AdminConsole–> Home–>Deployment Summary–> (click on your WebApp)–>Monitoring (Tab)—> WorkLoad (Sub-Tab)

    Log in to Reply
  • Joy SenSharma
    January 24th, 2010 on 3:37 pm

    Hi Jim,

    I have published a New Post … On how to assign a Global WorkManager to WebApplication…May be it will be helpful.

    http://jaysensharma.wordpress.com/2010/01/24/workmanager-at-webapplication-level/

    Thanks
    Jay SenSharma

    Log in to Reply
• Sharma Jampani
  February 17th, 2010 on 5:59 pm

  Hi Jay,

  Thanks for your reply on Oracle Forums.

  I just want to excute the solution that you have suggested for the log files… 🙂

  It was very great feeling when going through the blog.

  Thanks a lot for your knowledge sharing stuff..

  Regards
  Sharma Jampani

  Log in to Reply
• Hari
  March 8th, 2010 on 10:26 pm

  Hi Jay,

  Specifying work manager in weblogic.xml file does not work on weblogic 10.3.

  Please advise what changes are needed on 10.3 weblogic to set up work manager.

  Thanks
  Hari

  Log in to Reply
  • Joy SenSharma
    March 9th, 2010 on 2:38 am

    Hi Hari,

    Can you please elaborate little bit more. Like How do you see that the WorkManagers are not working? How you have configured the WorkManager?
    Can you please show us the “weblogic.xml” which you used to define/associate the Work Manager.
    Are you using ApplicationLevel Work Manager OR Global WorkManager ?

    Keep Posting 🙂
    THanks
    Jay SenSharma

    Log in to Reply
• Chaitanya Kiran PVN
  March 9th, 2010 on 6:39 am

  hi jay,

  Iam using the weblogic 10.3 server for my applocation. when i am accessing the site from the application some times it throws 503 error. iam not using the workmanager so it weblogic taking the default settings.please can you tell me how to over come this problem.

  thanks
  chaitanya

  Log in to Reply
  • Joy SenSharma
    March 9th, 2010 on 9:34 am

    Hi Chaitanya,

    Try tunning Accept Backlog Parameter:

    You can tune the number of connection requests that a WebLogic Server instance will accept before refusing additional requests. The Accept Backlog parameter specifies how many Transmission Control Protocol (TCP) connections can be buffered in a wait queue. This fixed-size queue is populated with requests for connections that the TCP stack has received, but the application has not accepted yet.

    You can tune the number of connection requests that a WebLogic Server instance will accept before refusing additional requests. For more information on TCP tuning, see Basic OS Tuning Concepts.

    1. If you have not already done so, in the Change Center of the Administration Console, click Lock & Edit.
    2. In the left pane of the console, expand Environment > Servers.
    3. On the Summary of Servers page, select the server instance for which you will configure connection backlog buffering.
    4. Expand the Configuration > Tuning tab.
    5. Modify the Accept Backlog value as necessary to tune the number of TCP connections the server instance can buffer in the wait queue.

    * If many connections are dropped or refused at the client, and no other error messages are on the server, the Accept Backlog value might be set too low.
    * If you are getting “connection refused” messages when you try to access WebLogic Server, raise the Accept Backlog value from the default by 25 percent. Continue increasing the value by 25 percent until the messages cease to appear.

    * The default value is 50 and the maximum value is operating system dependent.

    6. To activate these changes, in the Change Center of the Administration Console, click Activate Changes. Not all changes take effect immediately some require a restart.

    Log in to Reply
    • Chaitanya Kiran PVN
      March 9th, 2010 on 11:36 am

      hai joy,

      Thanks for your quick reply. iam already using the

      accept backlog = 300
      Socket Readers = 33

      thanks
      chaitanya

      Log in to Reply
      • Joy SenSharma
        March 9th, 2010 on 11:52 am

        Hi Chaitanya,
        Are you facing this issue 503-Service Unavailable…Only on Peak Load? Means i just wanted to confirm that How many users logged in to your application at peak load.
        Is there any Long running JDBC query / Long processable JMS messages inside your application?

        Thanks
        Jay SenSharma

        Log in to Reply
        • Chaitanya Kiran PVN
          March 9th, 2010 on 12:05 pm

          Hi Joy,

          yes iam facing the 503 service unavialble in the peak load.the number of users at the peak load 100 to 150 in my application we are using sockets for sending the xml only and we are using apache for bridge to weblogic.and we are not using jms for messaging. the user are login to the app through routing. 503 is return by the weblogic or the webserver(apache).
          thanks
          chaitanya
      • Joy SenSharma
        March 9th, 2010 on 12:13 pm

        Hi Chaitanya,

        AcceptBacklog = 300 SocketReaders = 33. You are using the Default values for these parameters. Whenever we get Connection Refuse we should first of all increast the AcceptBacklog parameter value to 25%.
        I think you should try that first. I am Just copy pasting the BEA document link here which also suggest the same thing…
        http://download.oracle.com/docs/cd/E13222_01/wls/docs90/perform/WLSTuning.html

        Please refer to the heading in above link “Tuning Connection Backlog Buffering”

        We should try that first…lets see what happens.

        Keep Posting 🙂
        Thanks
        Jay SenSharma

        Log in to Reply
        • Chaitanya Kiran PVN
          March 10th, 2010 on 4:48 am

          Hi Joy,

          yes iam facing the 503 service unavialble in the peak load.the number of users at the peak load 100 to 150 in my application we are using sockets for sending the xml only and we are using apache for bridge to weblogic.and we are not using jms for messaging. the user are login to the app through routing. 503 is return by the weblogic or the webserver(apache).
          thanks
          chaitanya
        • Joy SenSharma
          March 10th, 2010 on 5:14 am

          Hi Chaitanya,

          Still the First thing which i will suggest is to increase the Accept Backlog of WebLogic by 25% of the current value (300).
          As u have mentioned that u are using Apache as well so you can tune the MaxClient parameter as well…like MaxClient=256

          Set the MaxClients Directive to a High value
          http://download.oracle.com/docs/cd/E12839_01/web.1111/e10240/wls.htm

          You can let the HTTP Listener determine when to create more HTTPD processes. Therefore, set the MaxClients directive to a high value in the configuration file (httpd.conf). However, you need to consider the memory available on the system when setting this parameter.

          MaxClients=256 indicates that the listener can create up to 256 HTTPD processes to handle concurrent requests.

          If your HTTP requests come in bursts, and you want to reduce the time to start the necessary HTTPD processes, you can set MinSpareServers and MaxSpareServers (in httpd.conf) to have an appropriate number of processes ready. However, the default values of 5 and 10 respectively are sufficient for most sites.

          Keep Posting 🙂
          Thanks
          Jay SenSharma
• Sharma Jampani
  March 18th, 2010 on 2:49 pm

  Hi Jay,

  I have created global work managers for my application, i have set the min thread count to 200 and the max to 300, i am having 20 nodes and one admin server, weblogic server 10.3. I have got vip configured for the cluster as well.

  Now if i see the monitoring–>threads for one of the nodes, it is showing

  Active Execute Threads: 321
  Execute Thread count: 329
  Execute Thread Idle count: 18
  Queue length: 108
  Pending User Queue length: 108
  Hogging Thread Count: 292
  Standby Thread Count: 8

  All the other nodes are free, infact for most of the nodes, no thread was assigned.

  Application performance is really down and logging in itself is taking 7-8 mins on live.

  By seeing the above count, i assume that the load manager(vip) is not working properly and not forwarding the requests as expected? Am i right, could you please comment on this??

  Thanks in advance

  -Sharma Jampani

  Log in to Reply
  • Joy SenSharma
    March 18th, 2010 on 3:14 pm

    Hi Jampani,
    Just to isolate the issue …i have one question…Have you ever Shut down that node which is causing the problem and have u observed the same behavior with the next node?

    If after shutting down the Node-1 if you see that the same problem started happening with Node-2 …it clearly indicates that there is nothing wrong with the Nodes…rather we need to check the VIP configuration…

    Can you please isolate that where is the problem actually…with WebLogic Server Node or with the VIP? Then it will be very easy to debug this issue.

    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
    • Sharma Jampani
      March 18th, 2010 on 3:26 pm

      Hi Jay,

      Thanks for your response, actually day before yesterday, all the values that specified were seen for a node, yesterday, that node was free and a similar kind of thread count is applied to some other node…….

      as you have asked, yes it sometimes happening with one node and sometimes with other….

      so may be i think this might be a problme with the vip only.

      And, unfortunately, we don’t have access to see the vip configuration and its log… 🙁

      Thanks again, jay… 🙂

      Log in to Reply
      • Joy SenSharma
        March 18th, 2010 on 3:44 pm

        Hi Jampani,

        We need to confirm the type of Load balancing is selected in your VIP configuration….because there are basically four types of LoadBalancing modes are available…

        1. Round Robin
        Client requests are load balanced alternatively from one server to the next. Typically each server will get the same amount of users requests over time. This technique should be considered when servers have similar CPU power, memory and network access. Clients distribution fairness is ensured either by IP source address load balancing or connection load balancing (TCP/UDP established session).

        2. Ratio (Weighted Round Robin)
        Ratio works like round robin, but now each server has a weight associated to it in order to select a server upon its capacity. The Ratio algorithm is equivalent to Weighted Round Robin algorithm used in QoS techniques. Servers with more CPU processing power would be weighted more heavily, thus taking more requests over less weighted servers insuring better optimization in the load-balancing operation.

        3. Failover I doubt that in your case the VIP is configured in this mode
        In this method the load balancer will always select the first available server in a cluster list. When a selected server becomes unavailable (down, maintenance) then the load balancer will select the second one in this list. This algorithm is used primarily for backup mode servers.

        4. Least connections
        In this mode the load balancer sends new connections or new clients to the server that has the fewest number of connections. Two implementations exist today, one based on the load balancer table of connection history, one based on an agent sitting inside servers that reports server real activity. While the first implementation is similar to weighted round-robin, the second mode provides more accuracy about server connections, but requires additional software (agents) that could potentially hinder performance.

        So is it possible for you to confirm from your Network team to check which mode they have selected for ther VIP configuration…It looks like that there is definately some configuration mismatch in VIP.

        Keep Posting 🙂
        Thanks
        Jay SenSharma

        Log in to Reply
        • Sharma Jampani
          March 19th, 2010 on 10:56 am

          actually, there is a proxxy server involved in the architecture by the client and vip can’t see the actual client ip, vip is always seeing only the proxy ip for all the requests and so the ip seen is same for all the requests.

          still some discussions are going on and once i get some details i will ask for your help.. 🙂

          Thanks
• Sharma Jampani
  March 19th, 2010 on 10:42 am

  hi jay,

  can you please help me with the steps involved in implementing 2-way ssl on weblogic-10.3?

  Thanks in advance

  -Sharma Jampani

  Log in to Reply
  • Joy SenSharma
    March 19th, 2010 on 4:11 pm

    Hi Jampani,

    Here is a Post of one of My Best friend (Who is WebLogic Security Expert) ..you can refer to his post: http://secure-zone.blogspot.com/2009/10/configuring-two-way-ssl-between-client.html.
    For weblogic security related any issues i think he is the best. You can ask any doubt related to security in his forum.

    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
    • Sharma Jampani
      March 22nd, 2010 on 3:18 pm

      i have just put some steps which i followed to implement 2-way ssl, could you please verify them, i am also thinking to upgrade my blog regularly on weblogic facts…. i need your help in this regard..

      This is my blog:: http://sharmajampani.blogspot.com/

      Please tell me your comments on this…

      Thanks

      Log in to Reply
• mehak mehta
  March 25th, 2010 on 11:56 pm

  Hi Jay

  I want to assign a workmanager to a particular ejb component only in my deployed jar.Can u tell what is the correct process.
  Plus I do not want to use commonj api.Is it possible.
  I have tried by defining workmanager in weblogic-ejb-jar.xml and using dispatch policy tag in my weblogic-ejb-bean tag.
  It is showing the created wormanager in deployed.jar->monitoring->workload
  But the workmanager is not getting any request all requests are going to default workmanager.
  Please help

  Log in to Reply
  • Joy SenSharma
    March 26th, 2010 on 12:14 am

    Hi Mehak,
    How are you defining your WorkManager? Means Is it a Global WorkManager or a ApplicationLevel Defined WorkManager?
    Is it working for “Global WorkManager”? (Just to Isolate the issue can u please confirm)
    Which version of WLS you are using?

    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• Srikant
  April 2nd, 2010 on 4:57 am

  I am new to weblogic, i am unable to add user for my Human Workflow.
  the page doesn’t display anything. Also i noticed that the default SOA_SERVER1 and admin server are up and running while BAM_server1 is not running.

  pls help.

  Log in to Reply
  • Joy SenSharma
    April 2nd, 2010 on 4:15 pm

    Hi Srikant,
    Which Version of WebLogic you are using?
    Which kind of WorkFlow you are using and how exactly you are trying to add User to it? WorkFlow is a very new word for me. Are you talking about WorkShop PageFlows?
    As you have mentioned that BAM_server1 is not running…What is the entry in the Server Log… Are u seeing any Error or exception there?

    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• arpit
  April 29th, 2010 on 7:42 am

  hi jay,

  I have seen the delay in execution if we do not configure min and max thread for workmanager and i am using global work manager.

  So, is it essential to give these constraints. because when i configure them , the execution is going fine .

  I just wanted to know the difference .

  Thanks in advance

  Arpit b

  Log in to Reply
  • Joy SenSharma
    April 29th, 2010 on 11:14 am

    Hi Arpit,

    Hi,

    Few Pointes:

    1). Requests without an explicit dispatch policy use the default WorkManager of the application. This means that each application has its own default WorkManager that is not shared with other applications.

    2). Each servlet or RMI request is associated with a WorkManager. By default, all requests are associated with the application default WorkManager. The dispatch-policy element can be used to associate a request with a specific WorkManager either defined within an application scope or globally at the server level.

    3). Execute queues are always global whereas WorkManagers are always application scoped. Even WorkManagers defined globally in the console are application scoped during runtime. This means that each application gets into own runtime instance that is distinct from others, but all of them share the same characteristics like fair-share goals. This is done to track work at the application layer and to provide capabilities like graceful suspension of individual applications.
    
    Here is the Point….If you configure a Global WorkManager from AdminConsole WITHOUT assigning Any Constraints to it ….Like Max/Min Thread constraints (Then you can consider it as another default workmanager for your application)…then Your Application which is referring to it…will not get any way to ask WLS to Provided at least some minimum number of Threads always ready to process the requests (Bcoz of slef tunning feature applies in this case WLS will decide…Therad creation & Allocation)…WLS will decide which threads is required & which thread should be cleaned…

    But if you provide a Min Thread /Max Thread constraints to your Global Work manager then you are Instructing that At any point of time there should be at least that number of Minimum Threads should be always ready to process new requests…This improves the performance of your application.
    

    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
  • Joy SenSharma
    April 29th, 2010 on 11:15 am

    Hi Arpit,

    Few Pointes:

    1). Requests without an explicit dispatch policy use the default WorkManager of the application. This means that each application has its own default WorkManager that is not shared with other applications.

    2). Each servlet or RMI request is associated with a WorkManager. By default, all requests are associated with the application default WorkManager. The dispatch-policy element can be used to associate a request with a specific WorkManager either defined within an application scope or globally at the server level.

    3). Execute queues are always global whereas WorkManagers are always application scoped. Even WorkManagers defined globally in the console are application scoped during runtime. This means that each application gets into own runtime instance that is distinct from others, but all of them share the same characteristics like fair-share goals. This is done to track work at the application layer and to provide capabilities like graceful suspension of individual applications.
    
    Here is the Point….If you configure a Global WorkManager from AdminConsole WITHOUT assigning Max/Min Thread constraints (Then you can consider it as another default workmanager for your application)…then Your Application which is referring to it…will not get have any way to ask WLS to Provided at least some minimum number of Threads always ready to process the requests…WLS will decide which threads is required & which thread should be cleaned…

    But if you provide a Min Thread /Max Thread constraints to your Global Work manager then you are Instructing that At any point of tim there should be at least that number of Minimum Threads should be always ready to priocess new requests…This improves the performance of your application
    

    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• rajasekhar
  May 7th, 2010 on 12:47 pm

  can you explain what kind of error is this

  Compilation of JSP File ‘/index.jsp’ failed:
  index.jsp:179:16: com.bea.plateng cannot be resolved
  <% if (com.bea.plateng.common.util.RegistryCheck.isComponentInstalled(
  ^————^

  thanks
  raja

  Log in to Reply
• Tariq Khan
  June 9th, 2010 on 12:10 am

  Hi Jay,

  We configured Custom Work-manager .
  and successfully STUCK the WLS many thanks to you.
  Now the next part:
  We would like to see 51st request get rejected instead of waiting by the Managed Server.
  Can we achieve this using “work-manager-shutdown-trigger” .
  Let me know.

  Cheers.
  TK

  Log in to Reply
  • admin
    June 9th, 2010 on 12:53 am

    Hi TK,

    It’s good to hear that u are able to Make your Server as stuck by using servletContext.log(“a very very Long Message”)… for testing this feature.

    Now as u want that WebLogic server should not entertain 51st request …then u can set Capacity Constraing in your WorkManager….Like described in the below Link: http://middlewaremagic.com/weblogic/2010/01/24/workmanager-at-webapplication-level/ like following:

    <capacity>
    <name>CapacityConstraint-0</name>
    <target>AdminServer</target>
    <count>51</count>
    </capacity>

    You cannot achieve the same using “work-manager-shutdown-trigger”.
    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• khaleel
  June 26th, 2010 on 9:06 pm

  Hi,

  I would like to know, how to acheive the load balance using workmanager. We have 5 manage servers. We are using stateless session bean (EJB3.0) and default workmanager. Invoke the EJB from client side, the request come to one of manage server. For e.g. if request come to first manage server then it is always using the workmanager from same server.
  our requirement is to get the workmanager from different manage server in round robin fashion. Please let me know how to acheive this ? appreciate the sample code

  Thanks,
  Khaleel

  Log in to Reply
• rockss
  July 8th, 2010 on 5:28 pm

  Hi Joy,

  Need one info, Is it possible to redirect the console/(.out logs) output into application logs.

  Actully in our application plenty of debug messages are going inside console/.out files(bcoz of sysout or printstaktrace), is there any mechanism where we can redirect any message which is getting written from application using sysout/printstacktarce into the application logs on weblogic instead of console.

  Please suggest.

  Thanks

  Log in to Reply
  • JaySenSharma
    July 8th, 2010 on 5:59 pm

    Hi Rocks,

    Point – 1). In your Application donot use System.out.println(“Any Message”) Any where. Because these instructions will go insode the Standard Output device …means Console Out or the Out File.

    Point – 2). Whenever you write a try{} catch(Exception e){} Block inside your application….always make sure that you do not use
    try{
    //some code here
    }
    catch(Exception e){
    e.printStackTrace();
    }

    Rather you need to use some application logging support like Log4j to Log the relevant messages inside the Application Log …Like Following:
    try{
    //Something Wrong Goes Here
    }
    catch(Exception e){
    logger.log(“Useful Message in Application Log: “+e.getMessage());
    // Or something useful here to prevent the entries go in the Out file
    }

    NOTE: If your application is already ready and u dont have any option to change the Application Code then try this Use the following JAVA_OPTION to redirect the STDOUT entries of server to the Server Log .
    -Dweblogic.log.RedirectStdoutToServerLogEnabled=true
    Above JAVA_OPTION will force all the STDOUT entries to go inside the Server Log and not in a Separate Stdout Log.

    But there is no way to Tell the JVM to write it’s STDOUT Logs Entries inside the Application Log.
    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• KR
  October 15th, 2010 on 7:33 pm

  hi Jay,
  i am new to the forum and of course to WLS. I have a server with 4 managed instances running osb on top. I always get the 0 idle thread alerts from one or the other instance. Standby thread count stays between 8-10. How can i make tune so that i always have atleast one 1 idle thread all time. there is no much load on the server hardly two active requests all the time. Its 10.3.3 and my global min threads constraint has 40 threads in it. i appreciate your help.

  Thanks
  KR

  Log in to Reply
• KR
  October 15th, 2010 on 11:42 pm

  Hi jay I figured that out. I added a MEM ARG in startWeblogic.sh with min threadpool size and that, as i understand, should tune the default work manager.
  USER_MEM_ARGS=”-Dweblogic.threadpool.MinPoolSize=10″

  Log in to Reply
  • JaySenSharma
    October 16th, 2010 on 12:16 am

    Great…

    Log in to Reply
• sumit
  November 21st, 2010 on 11:46 pm

  Hi Jay,

  1) Will it possible to extend the default work manager?? What i mean i define a work manager with the name as “default” and define my own constrains . Then in this case it the WM will over ride the values for these constrains and rest all parameter will be default? Right?

  2) We all say that it always recommended to use WM rather than using native thread (java thread) ? But i want to know do you see any advantage of using native thread rather than using WM?? This just to know are there any advantage of native aover WM??

  3Above examples are very helpful for configuring the WM. Can we have one post which shows the real usefulness of WM in multithread environment.

  Thanks
  Sumit

  Log in to Reply
  • JaySenSharma
    November 22nd, 2010 on 12:10 am

    Hi Sumit,
    1).
    It is correct that if we want to override the default workmanager properties then we need to create a new workmanager with exactly the same name “default”. We can do any kind of customization/constraints with this workmanager now. What all parameter you will override for your own “default” your manager will use all those customized features …but for the Paramaters which u didnot modify for “default” workManager …then it will use the default values for those parameters.
    Example : Suppose The Original “default” workmanager was having 10 Parameters to be tuned. You created a new workManager with name “default” and overrided 5 Parameter values. In that case YOUR “default” workmanager will start using the 5 changes you made….along with the Default values of the ORIGINAL “default” workManager.

    2).
    WorkManagers are always better choice compared to the WebLogic 81 Style Execute Queues. NOTE: WorkManager Threads comes under the category of “Execute Threads” and these are not at all related to the “Socket Reader Threads”

    If you are asking about the difference between the Native Socket Readers Vs Pure Java Socket readers then it has nothing to do with the WorkManager or ExecuteQueue Threading model of WebLogic. Because WorkManagers are responsible for processing the requests…whereas the Socket Reader Threads are only responsible for Listening the Incoming clients requests. To know more about this Please refer to : Pure-Java Versus Native Socket Reader Implementations
    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• sumit
  November 22nd, 2010 on 11:16 am

  Jay,

  Presently our application is using spring framework. Thus we are using java threads.

  Now we want to migrate to WM so that we can take advantage of weblogic thread management . So how should we proceed in this direction?

  What i understand is
  1) Define our WM and define all the constrains and target each MS to this WM.Do we need to change any thing from code side? Do we need to perform any thing extra?

  This is the first time we are exploring on WM.So i need your help on this.

  Thanks
  Sumit

  Log in to Reply
  • JaySenSharma
    November 22nd, 2010 on 11:27 am

    Hi Sumit,
    To Use the WorkManagers we need not to make any Coding Changes inside our Application Code. You need to create either a Application level WorkManager or a Global WorkManager and then you need to refer to those workmanagers using the Deployment descriptors (weblogc.xml/ weblogic-ejb-jar.xml) of your Application. Workmanager Usage is independent from the Application Frameworks…So even if you use Hibernate/Swing/JSF/Struts…etc u need not to make any Application Code changes in order to use Workmanagers.
    Until u don’t use the JNI or Pure Native implementation Whatever Threads your Application creates it is “JAVA THREAD” only. so please don’t be confused with Java Thread and the Native Threads.
    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• sumit
  November 22nd, 2010 on 2:40 pm

  jay,

  This is my understanding from last post

  1) Create WM in admin console
  2) Refer that WM in weblogic.xml
  3) Start the application.
  Thus all the thread that application has implemented will be start using WM . Right?

  Then when should we use commonj API’s. These API provide
  mechanism equivalent to runnable interface like run method etc.

  Thanks
  Sumit

  Log in to Reply
• ritu_21
  December 18th, 2010 on 4:44 pm

  hi Jay,

  Please explain below para by adducing an example i am germinating weblogic admin but ur blog is quite captaviating and instigates me to keep practising new stuff and understand the concepts of weblogic :-

  This JSR is named “JSR-237 (http://www.jcp.org/en/jsr/detail?id=237)”. JSR-237 specifies the Work Manager API, which provides abstraction from the lower-level APIs that enable an application to access a container-managed thread. Work Manager API also provides a mechanism to join various concurrent work items, so an application can programmatically add the dependency of work completion as a condition for starting other tasks. This can be useful for implementing workflow types of application. These features were difficult to implement prior to Work Manager.
  thanks ritu,looking to hear from u.

  Log in to Reply
  • JaySenSharma
    December 18th, 2010 on 7:59 pm

    Hi Ritu,
    WorkManager JSR is invented by the joint effort by IBM and BEA Systems. Which allows an Application to access the Container managed Threads. Earlier to this specification it was not possible for an Application to access the Container managed Threads.

    Example: WebLogic has many “Execute Threads” which actuilly processes Clients requests. As soon as a Client sends a request to an Application one ExecuteThread is allocated by the container to process clients request. Now once the Thread started processing the Clients request (Suppose executing Servlets/JSPs), In this case the Servlet or JSPs doesnot have any control on the ExecuteThreads.
    But with Work Managers it is possible for Applications to concurrently execute different works, each and every part of work is called as “WorkItem“. The Specialty of the Work Items is “They gets executed out of the Container Managed Thread Pools. (If we will use java.lang.Threads to achieve the same then they won’t be separated by the Container Managed Thread Pool). Now with the hepl of WorkManagers the Applications can choose How Many Threads should be allocated for request processing…They can decids the Max-Capacity of Threads, Min-Capacity of Threads…etc.
    So Basically The Work Manager API provides a higher level of abstraction for concurrent programming than Old Style java.lang.Thread. According to the Specification The WorkManager can be bounded to the JNDI tree of a Server and can be accessed by the Applications to create and Execute their WorkItems. At the application level, each instance of WorkManager returns a WorkItem.
    you can also refer to the following link to get more Info on WorkManagers:
    http://download.oracle.com/docs/cd/E13222_01/wls/docs92/commonj/commonj.html
    Work – This interface allows you to run application code asynchronously. By creating a class that implements this interface, you can create blocks of code that can be scheduled to run at a specific time or at defined intervals. In other words, this is the “work” that is handled within the Work Manager API.
    WorkItem – After a Work instance has been submitted to a WorkManager, the WorkManager returns a WorkItem. This WorkItem is used determine the status of the completed Work instance.
    WorkListener – The WorkListener interface is a callback interface that provides communication between the WorkManager and the scheduled work defined within the Work instance.
    WorkEvent – A WorkEvent is sent to a WorkListener as Work is processed by WorkManager.
    RemoteWorkItem – The RemoteWorkItem interface is an extension of the WorkItem interface that allows work to be executed remotely. This interface allows serializable work to be executed on any member in a cluster.
    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• sumit
  December 18th, 2010 on 8:41 pm

  Jay,

  Can you please elaborate on this:

  ” The Specialty of the Work Items is “They gets executed out of the Container Managed Thread Pools. (If we will use java.lang.Threads to achieve the same then they won’t be separated by the Container Managed Thread Pool).”

  I have a question.
  1) Say we are creating a thread inside on application using java.lang.Thread . These thread comes under execute thread? and will be manageable by Work Manager?

  Thanks
  Sumit

  Log in to Reply
  • JaySenSharma
    December 18th, 2010 on 10:34 pm

    Hi Sumit,
    Please refer to the JSR-237 “http://www.jcp.org/en/jsr/detail?id=237“.
    Section 2: Request
    Paragraph (2.1)

    If you think that WebLogic WorkManager follows the JSR-237 then Yes answer for your question is “You did not read few more lines in my previous reply…”

    “(If we will use java.lang.Threads to achieve the same then they won’t be separated by the Container Managed Thread Pool)”
    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• ritu_21
  December 20th, 2010 on 12:50 am

  Thanks jay

  Log in to Reply
• ritu_21
  December 20th, 2010 on 4:13 pm

  HI There,

  Theres a problem occured in out env,seems that the number of maximum concurrent threads get stuck and takes long time to process,the step that we tried to fix and monitor it is to reduce the number of concurrect threds and monitor if it brings any improvement at present the max count for workmanager is 100 we brough tit down to 75 and are mo nitoring at present ,now i need to know if such kind of issures occur then what should be the best way to approach to the problem ,i m not sure if the wrokmanager is set up at global level and how can we identify if its at global level and application level

  Log in to Reply
• ritu_21
  December 20th, 2010 on 4:15 pm

  thread dumpps for the same issue

  — end of trace

  “[ACTIVE] ExecuteThread: ’57’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=207 idx=0x33c tid=8526 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x20f59850[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x20f59850[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’58’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=208 idx=0x340 tid=8527 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x20f5a360[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x20f5a360[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ‘119’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=1146 idx=0x354 tid=14283 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x240132c0[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x240132c0[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’59’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=214 idx=0x358 tid=8539 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x1840ac88[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x1840ac88[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’60’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=215 idx=0x35c tid=8540 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x1840b7a8[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x1840b7a8[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[ACTIVE] ExecuteThread: ’61’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=216 idx=0x360 tid=8541 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x1840c2c8[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x1840c2c8[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[ACTIVE] ExecuteThread: ’62’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=217 idx=0x364 tid=8542 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x1840cde8[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x1840cde8[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[ACTIVE] ExecuteThread: ’63’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=218 idx=0x368 tid=8543 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x1840d908[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x1840d908[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’64’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=219 idx=0x36c tid=8544 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x1840e428[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x1840e428[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’65’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=220 idx=0x370 tid=8545 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x1840ef48[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x1840ef48[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’66’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=221 idx=0x374 tid=8546 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x1840fa68[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x1840fa68[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’67’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=222 idx=0x378 tid=8547 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x18410588[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x18410588[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’68’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=223 idx=0x37c tid=8548 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x184110a8[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x184110a8[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’69’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=224 idx=0x380 tid=8549 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x18411bc8[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x18411bc8[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’70’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=225 idx=0x384 tid=8550 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x184126e8[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x184126e8[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[ACTIVE] ExecuteThread: ’71’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=226 idx=0x388 tid=8551 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x18413208[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x18413208[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’72’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=227 idx=0x38c tid=8552 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x18413d28[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x18413d28[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’73’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=228 idx=0x390 tid=8553 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x18414848[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x18414848[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’74’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=237 idx=0x3b4 tid=8579 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x2119a770[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x2119a770[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[ACTIVE] ExecuteThread: ’75’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=238 idx=0x3b8 tid=8580 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x2119b280[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x2119b280[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[ACTIVE] ExecuteThread: ’76’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=239 idx=0x3bc tid=8581 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x2119eee8[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x2119eee8[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’77’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=240 idx=0x3c0 tid=8582 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x2119f9f8[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x2119f9f8[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’78’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=241 idx=0x3c4 tid=8583 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x211a0508[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x211a0508[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[ACTIVE] ExecuteThread: ’80’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=244 idx=0x3cc tid=8587 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x1eb7ff30[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x1eb7ff30[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’81’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=245 idx=0x3d0 tid=8589 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x1eb42500[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x1eb42500[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[ACTIVE] ExecuteThread: ’79’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=243 idx=0x3d4 tid=8586 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x1eaf21b8[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x1eaf21b8[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[ACTIVE] ExecuteThread: ’82’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=246 idx=0x3d8 tid=8590 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/jms/client/JMSXASession@0xc98cf10[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/jms/client/JMSSession.waitForState(JMSSession.java:4290)
  at weblogic/jms/client/JMSSession.stop(JMSSession.java:1164)
  ^– Lock released while waiting: weblogic/jms/client/JMSXASession@0xc98cf10[fat lock]
  at weblogic/jms/client/JMSConnection.stop(JMSConnection.java:771)
  ^– Holding lock: weblogic/jms/client/JMSXAConnection@0xed22ce0[thin lock]
  at weblogic/jms/client/WLConnectionImpl.stop(WLConnectionImpl.java:714)
  at weblogic/ejb/container/internal/JMSConnectionPoller.stopJMSConnection(JMSConnectionPoller.java:1093)
  at weblogic/ejb/container/internal/JMSConnectionPoller.disconnect(JMSConnectionPoller.java:925)
  at weblogic/ejb/container/internal/MDConnectionManager.cancelConnectionPolling(MDConnectionManager.java:284)
  ^– Holding lock: weblogic/ejb/container/internal/JMSConnectionPoller@0x10221900[thin lock]
  at weblogic/ejb/container/manager/MessageDrivenManager.undeploy(MessageDrivenManager.java:607)
  at weblogic/ejb/container/deployer/EJBDeployer.deactivate(EJBDeployer.java:1895)
  at weblogic/ejb/container/deployer/EJBModule.doDeactivate(EJBModule.java:1067)
  at weblogic/ejb/container/deployer/EJBModule.deactivate(EJBModule.java:549)
  at weblogic/application/internal/flow/ModuleListenerInvoker.deactivate(ModuleListenerInvoker.java:124)
  at weblogic/application/internal/flow/DeploymentCallbackFlow$2.previous(DeploymentCallbackFlow.java:387)
  at weblogic/application/utils/StateMachineDriver.previousState(StateMachineDriver.java:52)
  at weblogic/application/utils/StateMachineDriver.previousState(StateMachineDriver.java:42)
  at weblogic/application/internal/flow/DeploymentCallbackFlow.deactivate(DeploymentCallbackFlow.java:88)
  at weblogic/application/internal/flow/DeploymentCallbackFlow.deactivate(DeploymentCallbackFlow.java:80)
  at weblogic/application/internal/BaseDeployment$2.previous(BaseDeployment.java:641)
  at weblogic/application/utils/StateMachineDriver.previousState(StateMachineDriver.java:52)
  at weblogic/application/utils/StateMachineDriver.previousState(StateMachineDriver.java:42)
  at weblogic/application/internal/BaseDeployment.deactivate(BaseDeployment.java:234)
  at weblogic/application/internal/DeploymentStateChecker.deactivate(DeploymentStateChecker.java:189)
  at weblogic/deploy/internal/targetserver/AppContainerInvoker.deactivate(AppContainerInvoker.java:99)
  at weblogic/deploy/internal/targetserver/BasicDeployment.deactivate(BasicDeployment.java:261)
  at weblogic/deploy/internal/targetserver/BasicDeployment.deactivateFromServerLifecycle(BasicDeployment.java:450)
  at weblogic/management/deploy/internal/DeploymentAdapter$1.doDeactivate(DeploymentAdapter.java:73)
  at weblogic/management/deploy/internal/DeploymentAdapter.deactivate(DeploymentAdapter.java:211)
  at weblogic/management/deploy/internal/AppTransition$6.transitionApp(AppTransition.java:66)
  at weblogic/management/deploy/internal/ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:233)
  at weblogic/management/deploy/internal/ConfiguredDeployments.deactivate(ConfiguredDeployments.java:198)
  at weblogic/management/deploy/internal/ConfiguredDeployments.undeploy(ConfiguredDeployments.java:191)
  at weblogic/management/deploy/internal/DeploymentServerService.shutdownApps(DeploymentServerService.java:188)
  at weblogic/management/deploy/internal/DeploymentServerService.shutdownHelper(DeploymentServerService.java:120)
  at weblogic/application/ApplicationService.halt(ApplicationService.java:142)
  at weblogic/t3/srvr/ServerServicesManager.haltInternal(ServerServicesManager.java:421)
  at weblogic/t3/srvr/ServerServicesManager.halt(ServerServicesManager.java:268)
  at weblogic/t3/srvr/T3Srvr.shutdown(T3Srvr.java:882)
  at weblogic/t3/srvr/T3Srvr.forceShutdown(T3Srvr.java:790)
  at weblogic/t3/srvr/ServerRuntime.forceShutdown(ServerRuntime.java:332)
  at weblogic/t3/srvr/ServerRuntime.shutdown(ServerRuntime.java:312)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  at jrockit/vm/Reflect.invokeMethod(Ljava/lang/Object;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Native Method)
  at sun/reflect/NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;(Native Method)
  at sun/reflect/NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  at java/lang/reflect/Method.invoke(Method.java:592)
  at weblogic/management/jmx/modelmbean/WLSModelMBean.invoke(WLSModelMBean.java:443)
  at com/sun/jmx/mbeanserver/DynamicMetaDataImpl.invoke(DynamicMetaDataImpl.java:213)
  at com/sun/jmx/mbeanserver/MetaDataImpl.invoke(MetaDataImpl.java:220)
  at com/sun/jmx/interceptor/DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:815)
  at com/sun/jmx/mbeanserver/JmxMBeanServer.invoke(JmxMBeanServer.java:784)
  at weblogic/management/jmx/mbeanserver/WLSMBeanServerInterceptorBase$16.run(WLSMBeanServerInterceptorBase.java:449)
  at jrockit/vm/AccessController.doPrivileged(AccessController.java:255)
  at jrockit/vm/AccessController.doPrivileged(AccessController.java:269)
  at weblogic/management/jmx/mbeanserver/WLSMBeanServerInterceptorBase.invoke(WLSMBeanServerInterceptorBase.java:447)
  at weblogic/management/jmx/mbeanserver/WLSMBeanServerInterceptorBase$16.run(WLSMBeanServerInterceptorBase.java:449)
  at jrockit/vm/AccessController.doPrivileged(AccessController.java:255)
  at jrockit/vm/AccessController.doPrivileged(AccessController.java:269)
  at weblogic/management/jmx/mbeanserver/WLSMBeanServerInterceptorBase.invoke(WLSMBeanServerInterceptorBase.java:447)
  at weblogic/management/mbeanservers/internal/SecurityInterceptor.invoke(SecurityInterceptor.java:443)
  at weblogic/management/jmx/mbeanserver/WLSMBeanServer.invoke(WLSMBeanServer.java:310)
  at weblogic/management/mbeanservers/compatibility/internal/RemoteMBeanServerImpl.invoke(RemoteMBeanServerImpl.java:203)
  at weblogic/management/mbeanservers/compatibility/internal/RemoteMBeanServerImpl_WLSkel.invoke(ILweblogic/rmi/spi/InboundRequest;Lweblogic/rmi/spi/OutboundResponse;Ljava/lang/Object;)Lweblogic/rmi/spi/OutboundResponse;(Unknown Source)
  at weblogic/rmi/internal/BasicServerRef.invoke(BasicServerRef.java:591)
  at weblogic/rmi/internal/BasicServerRef$1.run(BasicServerRef.java:482)
  at weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
  at weblogic/security/service/SecurityManager.runAs(Lweblogic/security/acl/internal/AuthenticatedSubject;Lweblogic/security/acl/internal/AuthenticatedSubject;Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;(Unknown Source)
  at weblogic/rmi/internal/BasicServerRef.handleRequest(BasicServerRef.java:478)
  at weblogic/rmi/internal/BasicServerRef.access$300(BasicServerRef.java:62)
  at weblogic/rmi/internal/BasicServerRef$BasicExecuteRequest.run(BasicServerRef.java:1040)
  at weblogic/work/ExecuteThread.execute(ExecuteThread.java:200)
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:172)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ‘110’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=341 idx=0x3dc tid=9359 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x21237f50[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x21237f50[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ‘109’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=340 idx=0x3e0 tid=9358 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x21238c30[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x21238c30[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ‘111’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=342 idx=0x3e4 tid=9360 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x212371b0[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x212371b0[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[ACTIVE] ExecuteThread: ’83’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=252 idx=0x3f0 tid=8614 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x1755d458[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x1755d458[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’84’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=253 idx=0x3f4 tid=8615 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x1755e348[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x1755e348[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[ACTIVE] ExecuteThread: ’85’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=254 idx=0x3f8 tid=8616 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x175623a0[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x175623a0[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[ACTIVE] ExecuteThread: ’86’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=255 idx=0x3fc tid=8617 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x17562ec0[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x17562ec0[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  “[STANDBY] ExecuteThread: ’87’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=256 idx=0x400 tid=8618 prio=5 alive, in native, waiting, daemon
  — Waiting for notification on: weblogic/work/ExecuteThread@0x175656e8[fat lock]
  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
  at java/lang/Object.wait(J)V(Native Method)
  at java/lang/Object.wait(Object.java:474)
  at weblogic/work/ExecuteThread.waitForRequest(ExecuteThread.java:156)
  ^– Lock released while waiting: weblogic/work/ExecuteThread@0x175656e8[fat lock]
  at weblogic/work/ExecuteThread.run(ExecuteThread.java:177)
  at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
  — end of trace

  Blocked lock chains
  ===================
  Chain 2:
  “ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” id=35 idx=0x98 tid=1933 waiting for java/lang/String@0x1c9265f8 held by:
  “ExecuteThread: ‘6’ for queue: ‘weblogic.socket.Muxer'” id=40 idx=0xac tid=1938 in chain 1

  Chain 3:
  “ExecuteThread: ‘2’ for queue: ‘weblogic.socket.Muxer'” id=36 idx=0x9c tid=1934 waiting for java/lang/String@0x1c9265f8 held by:
  “ExecuteThread: ‘6’ for queue: ‘weblogic.socket.Muxer'” id=40 idx=0xac tid=1938 in chain 1

  Chain 4:
  “ExecuteThread: ‘3’ for queue: ‘weblogic.socket.Muxer'” id=37 idx=0xa0 tid=1935 waiting for java/lang/String@0x1c9265f8 held by:
  “ExecuteThread: ‘6’ for queue: ‘weblogic.socket.Muxer'” id=40 idx=0xac tid=1938 in chain 1

  Chain 5:
  “ExecuteThread: ‘4’ for queue: ‘weblogic.socket.Muxer'” id=38 idx=0xa4 tid=1936 waiting for java/lang/String@0x1c9265f8 held by:
  “ExecuteThread: ‘6’ for queue: ‘weblogic.socket.Muxer'” id=40 idx=0xac tid=1938 in chain 1

  Chain 6:
  “ExecuteThread: ‘5’ for queue: ‘weblogic.socket.Muxer'” id=39 idx=0xa8 tid=1937 waiting for java/lang/String@0x1c9265f8 held by:
  “ExecuteThread: ‘6’ for queue: ‘weblogic.socket.Muxer'” id=40 idx=0xac tid=1938 in chain 1

  Chain 7:
  “ExecuteThread: ‘7’ for queue: ‘weblogic.socket.Muxer'” id=41 idx=0xb0 tid=1939 waiting for java/lang/String@0x1c9265f8 held by:
  “ExecuteThread: ‘6’ for queue: ‘weblogic.socket.Muxer'” id=40 idx=0xac tid=1938 in chain 1

  Chain 8:
  “ExecuteThread: ‘8’ for queue: ‘weblogic.socket.Muxer'” id=42 idx=0xb4 tid=1940 waiting for java/lang/String@0x1c9265f8 held by:
  “ExecuteThread: ‘6’ for queue: ‘weblogic.socket.Muxer'” id=40 idx=0xac tid=1938 in chain 1

  Open lock chains
  ================
  Chain 1:
  “ExecuteThread: ‘0’ for queue: ‘weblogic.socket.Muxer'” id=34 idx=0x94 tid=1932 waiting for java/lang/String@0x1c9265f8 held by:
  “ExecuteThread: ‘6’ for queue: ‘weblogic.socket.Muxer'” id=40 idx=0xac tid=1938 (active)

  Log in to Reply
• ritu_21
  December 20th, 2010 on 4:40 pm

  Also I need to know when we do kill -3 (process id)

  whers is the thread dump generated is it in the server logs..?

  Log in to Reply
  • JaySenSharma
    December 20th, 2010 on 6:56 pm

    Hi Ritu,
    When we issue the kill -3 in that case by default the Thread dump gets generated in the STDOUT of the Server. STDOUT is actually the output which we see in the Shell Prompt where we started the Server…
    Example 1): if you run your WLS like following then the Out put of the Shell prompt will generate the Thread Dump when u trigger the Kill -3 PID
    ./startWebLogic.sh

    Example 2). If you start your Server like following then the Thread Dump will be enerated in the file: “nohup.out ”
    nohup startWebLogic.sh > nohup.out

    The best way will be to collect the Thread Dump using “jstack” utility so the Thread Dump will be generated in the same window where you run the Jskack command …Please refer to : http://middlewaremagic.com/weblogic/?p=2281
    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• ritu_21
  December 22nd, 2010 on 6:47 pm

  Hi jay,

  In our env we have two tiers

  1.web chaneel(provides user interface)
  2.bls (actually process the reuest)

  at the moment we have 12 bls and 12 webchanells
  The BLS is capacity is 1000 (execute threads + execute queue)
  and webchannel is 600 so at high loads we see that the web chaneels reuests are queying up
  threads are waiting

  sp we thought of increasing the WC but the risk is
  can the WC execute 100 parallel requests before it runs out of memory/cpu ?

  what do u reconnd for such situation is dere any fix we can try at eithe rof tiers or apache
  as we have apache tier in between

  and if we increase the WC capacity what would happen at off peak loads

  cheers
  Ritu

  Log in to Reply
  • JaySenSharma
    December 22nd, 2010 on 7:50 pm

    Hi Ritu,
    Your Question is very much specific to your environment, And it is very difficult for an external guy to directly suggest some tuning on that environment without havind a complete understanding/Functionality and workflow of that environment.
    Even the approximate / correct tuning values can be suggested/figured out only during the Load Test. So i cannot exactly tell you what will happen at the peak load. Also 100 or 200 or 300 Parallel requests or any number of Concurrent requests never decides what will be the load on the CPU or on the Heap rather it depends on What task each and every threads is going to perform…..Exaan mple: It is possible that a single Thread c creates 20MB of Objects or consumes 40% of CPU cycles to process a request. So it totally depends on “What kind of operatins and requests these Threads are going to serve “.
    Thats why it becomes for an external person to suggest a tuning value or impact of some tuning parameter changes.
    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• ravibopara
  January 11th, 2011 on 4:36 pm

  Hi Jay,

  I have some Questions over the parameter self-tuning-thread-pool-size-max in config.xml

  I set this value to 70.

  During LoadTesting We found that the number of Pending requests where increasing however the Number of threads did not reach the max value of 70 it was somewhere around 56 and did not increase after that.

  Please let me know if the explanation was not clear

  Thanks
  Ravi

  Log in to Reply
  • Ravish Mody
    January 11th, 2011 on 7:23 pm

    Hi Ravi,

    self-tuning-thread-pool-size-max parameters grows and shrinks automatically as required, the server will add threads only if it improves throughput. If historical throughput statistics indicate that a higher thread count increased throughput, WebLogic increases the thread count. These measurements are taken every 2 seconds and the decision to increase or decrease the thread count is based on the current throughput measurement versus past values.

    Now in your case it seems that WLS dint find if by increasing the thread would improves the throughput which was based on the last throughput statistics. So would suggest you to try it again and give more load and see if it reaches the limit or else it seems to be a bug, but before saying that we would have to test it few times.

    Regards,
    Ravish Mody

    Log in to Reply
• ravibopara
  January 11th, 2011 on 9:23 pm

  Hi Ravi,

  At one point I could see more than 150 requests pending.

  Thanks
  Ravi

  Log in to Reply
  • JaySenSharma
    January 11th, 2011 on 11:17 pm

    Hi Ravi,
    Please refer to the following Post where it is mentioned regarding the SocketReaderThreads…http://middlewaremagic.com/weblogic/?p=4375

    WebLogic Socket Reader Threads: These threads listens to the incoming clients requests, Means basically these Threads deals with the Network traffic part. These are basically a percentage of Execute Threads. The Default ration between is Execute Threads and Socket Reader Threads is 33%. Means 33% Threads will be Socket Readers as default. Which can be tuned according to the requirement.
    Example: if we have 15 Execute Threads in Development Mode of WLS Server then it simply means that around 33% of these threads will work as Socket Reader Threads and rest Threads will be processing the Clients request.

    So in your case the “thread-pool-size-max” is 70 in that case it is common that around 30-33% threads will work as Socket reader threads and remaining will work as Worker Execute Threads.
    So if 70% execute threads are already working (processing clients request) then it means that 150 Additional request need to be in the queue….It means your environment is not setup properly for the Peak Load or The application is taking longer time to process requests.
    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• sumit
  January 14th, 2011 on 11:07 am

  Jay,
  Is it possible to configure WM for each type servlet( i.e each type of http request) . We are defining 2 types of servlet classes in web.xml that will handle the request depending on the type of url pattern.

  Thus we are thinking to have WM for each type of servlet (i.e request ).

  Thanks
  Sumit

  Log in to Reply
  • JaySenSharma
    January 14th, 2011 on 2:27 pm

    Hi Sumit,
    At individual Servlet level you can assign workmanager like following ….edit the “web.xml” like following:

    <servlet>
    <init-param>
    <param-name>wl-dispatch-policy</param-name>
    <param-value>YourWorkManager</param-value>
    </init-param>
    .
    .
    .
    
    </servlet>
    

    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• sumit
  January 14th, 2011 on 3:41 pm

  Jay,

  Then in this case i don’t need to define anything in weblogic.xml.Correct?

  I will create 2 WM from admin console and provide the WM reference in web.xml as mentioned above.

  Thanks
  Sumit

  Log in to Reply
  • Ravish Mody
    January 14th, 2011 on 5:33 pm

    Hi Sumit,

    Yes, your understating is correct.

    Regards,
    Ravish Mody

    Log in to Reply
• PabloRS
  March 26th, 2011 on 2:12 am

  Hi, i have a question.
  My project have ten Webservices (JAX-WS) running on Weblogic 10.3.3
  My need is control the threads for each service.
  I will use the Workmanager to control this.
  I think create one Workmanager for each service.

  For testing purpose i create by weblogic console one
  workmanager named Test.

  In my code on the Implementation Class for my services i try to call
  them using this:
  @Resource(mappedName=”java:comp/env/wm/Test”, authenticationType=AuthenticationType.CONTAINER,
  type=commonj.work.WorkManager.class,
  name=”Test”)

  In my class the Webservices look-like this:

  @Autowired
  private fooBO fooBO;

  @WebMethod
  @SOAPBinding(parameterStyle = ParameterStyle.BARE)
  @WebResult(name = “FooResponse”)
  public FooResponse getAllTheNumbers (
  @WebParam(name = “getAllTheNumbers”) FooRequest request)
  throws BusinessFault, InfrastructureFault {

  return fooBO.getAllTheNumbers(request);
  }

  Not warning is showed but when my application runs all calls still using the
  default workmanager.

  My question is:
  How to set each Webservice to use only the defined Workmanager.
  How to do this, using annotations if possible.

  I use Spring 3 and WebLogic 10.3.3. I create a Jython script to create the
  workmanagers based in the script found here.
  My problem is force the WS use my own WorkManager.

  Thanks! Great Site!

  Log in to Reply
  • JaySenSharma
    March 26th, 2011 on 1:19 pm

    Hi PabloRS,
    The annotation @Resource entries will be resolved in the run time and if will be reflected in “web.xml” dynamically on the fly when the Annotation will be processed. But apart from this you must have “weblogic.xml” file as well inside your WebService “WEB-INF/” directory with the following tag:

    <wl-dispatch-policy>WorkManagerA</wl-dispatch-policy>
    As described in the following link: http://middlewaremagic.com/weblogic/?p=385

    OR You can use the plan.xml feature to dynamically associate a WorkManager to your WebService. As described in the following link:
    http://middlewaremagic.com/weblogic/?p=687
    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• PabloRS
  April 1st, 2011 on 6:34 pm

  I use the the annotation to make the reference to my Work Managers
  and my test was as follow:

  Create 2 Workmanagers on WL Console (Test1 and Test2).
  Both using the same constraints max and min threads.

  Put the follow Annotations on my two Implemented Class for my webservices (Is it the rigth place??)

  @Resource(mappedName=”java:comp/env/wm/Test1”, authenticationType=AuthenticationType.CONTAINER,
  type=commonj.work.WorkManager.class,
  name=”Test1”)

  @Resource(mappedName=”java:comp/env/wm/Test2”, authenticationType=AuthenticationType.CONTAINER,
  type=commonj.work.WorkManager.class,
  name=”Test2”)

  At all the reference works, because if i use an inexistent name it will give me a warning message on log.

  After that i put on weblogic.xml the follow:

  Test1
  Test2

  No one warning message, or error message appears. On Workload from my deployment i see the both Workmanagers and the default one too.

  BUT, when i call both of my webservices both are using the Test1 WM.. I set the annotation for them but always the Test1 is used.

  I think this is order on the weblogic.xml i test switching the tags as follow:

  Test2
  Test1

  And now all my services are using the Test2 one.

  1. The annotation alone dont do nothing. It points to Workmanager but.. nothing happen
  I need to create a variable and use it for call my services? ??

  2. Have an form to define and “wl-dispatch-policy” for each component by my deploy? One for each Webservice i have??

  Thanks again!

  Great site. Please help me , several days on it without progress..

  Log in to Reply
• PabloRS
  April 1st, 2011 on 6:37 pm

  When it as shown:
  Test1
  Test2
  Really are:
  “wl-dispatch-policy”Test1″/wl-dispatch-policy”
  “wl-dispatch-policy”Test2″/wl-dispatch-policy”

  The tags on weblogic.xml (The tags are removed by the site.)

  Log in to Reply
• Appi
  June 14th, 2011 on 9:15 am

  Hi Jay ,

  We have created a workmanager with Min and Max Thread constraint and associated it with the MQSeriesAdpater application deployed in OracleSOASuite installation.

  However, under load, we do not see the requests being processed by the work manager in the console. Please let us know if any additional configuration is required or not for the work manager settings.

  We are using WLS 10.3.3 and using Global WM .

  Thanks ,
  Appi

  Log in to Reply
  • JaySenSharma
    June 14th, 2011 on 4:26 pm

    Hi Appi,
    If the workManager works fine under normal load and Not Working in Heavy Load then …it should be a BUG.
    We must contact Oracle Support, because there is no additional configuration needed to make the WorkManager work differently on Peak Load or on Normal Load.
    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• Appi
  June 15th, 2011 on 9:26 am

  Hi Jay ,

  Thanks for your inputs.

  Actually , The work manager configuration is not working at all, irrespective of the load.

  For now i have defined the WM in Config.xml file.

  Please suggest if we need to configure WM as Application-scoped Work Managers for MQSeriesAdpater application.

  Thanks,
  Appi

  Log in to Reply
• Appi
  June 17th, 2011 on 9:46 am

  Hi Jay ,

  We wanted to configure Workmanager for MQSeries Adapter which comes with the SOA suite.

  For configuring workmanager , we need to use wl-dispatch-policy parameter to link workmanager .

  here , i need to know — which xml file we should use to make this linking ..

  Please let me know the xml file or configuration file for MQSeries Adapter.

  Thanks,
  Appi

  Log in to Reply
  • JaySenSharma
    June 17th, 2011 on 2:17 pm

    HI Appi,
    WorkManager can be associated to a WAR file using” weblogic.xml”

    <?xml version=”1.0″ encoding=”UTF-8″?>
    <weblogic-web-app xmlns=”http://www.bea.com/ns/weblogic/90″>
    <wl-dispatch-policy>myConsoleWorkManager</wl-dispatch-policy>
    </weblogic-web-app>

    If you have an EJB jar file then using “weblogic-ejb-jar.xml” file also you can associate the WorkManager to your EJB Components using <dispatch-policy> tag like following:

    <?xml version=’1.0′ encoding=’UTF-8′?>
    <weblogic-ejb-jar xmlns=”http://www.bea.com/ns/weblogic/10.0″ xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”>
    <weblogic-enterprise-bean>
    <ejb-name>CalculatorBean</ejb-name>
    <stateless-session-descriptor>
    <pool></pool>
    <stateless-clustering>
    <home-is-clusterable>false</home-is-clusterable>
    <stateless-bean-is-clusterable>false</stateless-bean-is-clusterable>
    </stateless-clustering>
    </stateless-session-descriptor>
    <transaction-descriptor></transaction-descriptor>
    <jndi-name>X</jndi-name>
    <dispatch-policy>JackWorkManager</dispatch-policy>
    <remote-client-timeout>0</remote-client-timeout>
    </weblogic-enterprise-bean>
    </weblogic-ejb-jar>

    At an individual Servlet level you can associate WorkManager like following inside your “web.xml” file:

    <servlet>
    <servlet-name>VerySlowJSP</servlet-name>
    <jsp-file>veryslow.jsp</jsp-file>
    <init-param>
    <param-name>wl-dispatch-policy</param-name>
    <param-value>myCustomWorkManager</param-value>
    </init-param>
    </servlet>

    I am not aware about any other techniqueue to associate a WorkManager with an Application.
    .
    .
    Thanks
    Jay SenSharma

    Log in to Reply
• felmerien
  September 19th, 2011 on 8:56 pm

  Hi,

  have you some experience return on the max number of thread WLS 10 can manage without problem?
  I try to explain: with WLS 8.1 and execute queue it was hard to tune the good number of thread but possible (with some processor and for an application we find that after 30 concurrent thread, there’s lower performance because of switch on processeur, for others it was 45 for example)
  Now i can see sometimes more than 400 thread on one JVM and it seem to be very much for me!!
  When we have problem of response time in PRODUCTION, i can see sometimes more than 100 Stuck thread on a resource, so i think it’s the reason of bad response time but with WM i see the number of thread growth and the CPU is not very important (less than 50%) because the stuckthread are do nothing (waiting for resource)
  So i don’t understand the good tuning to apply since WorkManager…ok it’s not normal to have a lot of stuck thread but if there’s no matter about the max number of thread, is it useful to stop and restart managed server?
  do you think we can have have bad performance if there’s a lot of thread?
  thank you for your experience return and sorry for my poor english (i’m froggy ^^)

  Log in to Reply
• Renne
  September 20th, 2011 on 11:37 am

  In WlS 9.x version onwards weblogic uses the concept of “Work manager”. by default the max threads capacity in work manager is “Unlimited”. Based on the requests, it will create the threads and it is self tunable. if you want to tune the max threads in work manager , you can go for the max threads constraint

  But you have to look into stuck threads, why that particular operation, on which stuck thread is working, is taking long time and take necessary steps accordingly.Other wise the server performance would be slow as the request is taking long time.

  Log in to Reply
• Aravindh
  December 19th, 2011 on 7:15 am

  Hi jay,

  need an help, in our production environment, we have two managed servers- clustered for an application
  We have EM monitoring enabled on those managed servers
  We are seeing the following alert on both the managed servers
  “The number of work manager stuck threads is 11”

  The managed servers are in warning state because of the stukc threads, have takem the thread dump, please let me know if you could help me in analyzing it? thanks

  could ypu please help me as how should i be proceeding on this alert?

  Log in to Reply
• Aravindh
  December 19th, 2011 on 9:49 am

  “[STUCK] ExecuteThread: ’14’ for queue: ‘weblogic.kernel.Default (self-tuning)'” id=219 idx=0x368 tid=31746 prio=1 alive, waiting, native_blocked, daemon

  — Waiting for notification on: oracle/adfinternal/controller/util/RequestLock@0x47ad09c0[fat lock]

  at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)

  at java/lang/Object.wait(J)V(Native Method)

  at java/lang/Object.wait(Object.java:485)

  at oracle/adfinternal/controller/util/RequestLock.lock(RequestLock.java:42)

  ^– Lock released while waiting: oracle/adfinternal/controller/util/RequestLock@0x47ad09c0[fat lock]

  at oracle/adfinternal/controller/state/RootViewPortContextImpl.lockViewPortRequestLock(RootViewPortContextImpl.java:604)

  at oracle/adfinternal/controller/state/ControllerState.initializeRequest(ControllerState.java:833)[inlined]

  at oracle/adfinternal/controller/state/ControllerState.initializeRequest(ControllerState.java:754)[optimized]

  at oracle/adfinternal/controller/application/AdfcConfigurator.beginRequest(AdfcConfigurator.java:50)

  at org/apache/myfaces/trinidadinternal/config/GlobalConfiguratorImpl._startConfiguratorServiceRequest(GlobalConfiguratorImpl.java:562)[inlined]

  at org/apache/myfaces/trinidadinternal/config/GlobalConfiguratorImpl.beginRequest(GlobalConfiguratorImpl.java:212)[optimized]

  at org/apache/myfaces/trinidadinternal/webapp/TrinidadFilterImpl.doFilter(TrinidadFilterImpl.java:155)[optimized]

  at org/apache/myfaces/trinidad/webapp/TrinidadFilter.doFilter(TrinidadFilter.java:92)[optimized]

  at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]

  at oracle/mos/mobile/auth/AuthorizationFilter.doFilter(AuthorizationFilter.java:110)[optimized]

  at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]

  at oracle/security/jps/ee/http/JpsAbsFilter$1.run(JpsAbsFilter.java:111)[optimized]

  at jrockit/vm/AccessController.doPrivileged(AccessController.java:254)[inlined]

  at oracle/security/jps/util/JpsSubject.doAsPrivileged(JpsSubject.java:313)[inlined]

  at oracle/security/jps/ee/util/JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:413)[inlined]

  at oracle/security/jps/ee/http/JpsAbsFilter.runJaasMode(JpsAbsFilter.java:94)[inlined]

  at oracle/security/jps/ee/http/JpsAbsFilter.doFilter(JpsAbsFilter.java:161)[optimized]

  at oracle/security/jps/ee/http/JpsFilter.doFilter(JpsFilter.java:71)[optimized]

  at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]

  at oracle/security/jps/ee/http/JpsAbsFilter$1.run(JpsAbsFilter.java:111)[optimized]

  at jrockit/vm/AccessController.doPrivileged(AccessController.java:254)[inlined]

  at oracle/security/jps/util/JpsSubject.doAsPrivileged(JpsSubject.java:313)[inlined]

  at oracle/security/jps/ee/util/JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:413)[inlined]

  at oracle/security/jps/ee/http/JpsAbsFilter.runJaasMode(JpsAbsFilter.java:94)[inlined]

  at oracle/security/jps/ee/http/JpsAbsFilter.doFilter(JpsAbsFilter.java:161)[optimized]

  at oracle/security/jps/ee/http/JpsFilter.doFilter(JpsFilter.java:71)[optimized]

  at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]

  at oracle/dms/servlet/DMSServletFilter.doFilter(DMSServletFilter.java:136)[optimized]

  at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[optimized]

  at weblogic/servlet/internal/RequestEventsFilter.doFilter(RequestEventsFilter.java:27)[optimized]

  at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:56)[inlined]

  at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3715)[inlined]

  at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3681)[optimized]

  at weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)[optimized]

  at weblogic/security/service/SecurityManager.runAs(SecurityManager.java:120)[inlined]

  at weblogic/servlet/internal/WebAppServletContext.securedExecute(WebAppServletContext.java:2277)[inlined]

  at weblogic/servlet/internal/WebAppServletContext.execute(WebAppServletContext.java:2183)[optimized]

  at weblogic/servlet/internal/ServletRequestImpl.run(ServletRequestImpl.java:1454)[optimized]

  at weblogic/work/ExecuteThread.execute(ExecuteThread.java:207)[inlined]

  at weblogic/work/ExecuteThread.run(ExecuteThread.java:176)[optimized]

  at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)

  — end of trace

  Blocked lock chains

  ===================

  Chain 2:

  “ExecuteThread: ‘2’ for queue: ‘weblogic.socket.Muxer'” id=28 idx=0x7c tid=21613 waiting for java/lang/String@0x46496e90 held by:

  “ExecuteThread: ‘0’ for queue: ‘weblogic.socket.Muxer'” id=26 idx=0x74 tid=21611 in chain 1

  Chain 3:

  “ExecuteThread: ‘3’ for queue: ‘weblogic.socket.Muxer'” id=29 idx=0x80 tid=21614 waiting for java/lang/String@0x46496e90 held by:

  “ExecuteThread: ‘0’ for queue: ‘weblogic.socket.Muxer'” id=26 idx=0x74 tid=21611 in chain 1

  Open lock chains

  ================

  Chain 1:

  “ExecuteThread: ‘1’ for queue: ‘weblogic.socket.Muxer'” id=27 idx=0x78 tid=21612 waiting for java/lang/String@0x46496e90 held by:

  “ExecuteThread: ‘0’ for queue: ‘weblogic.socket.Muxer'” id=26 idx=0x74 tid=21611 (active)

  Log in to Reply
  • JaySenSharma
    December 20th, 2011 on 1:00 am

    Hi Aravindh,

    Can you send us the complete thread dump of the time when the issue occurred in the E-Mail address contact@middlewaremagic.com. Whenever we collect thread dumps we should make sure that we collect at least 5-6 thread dumps each collected in the interval of 9-10 seconds.

    Also do let us know which version of WLS server and ADF libraries your application is using?
    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• Aravindh
  December 20th, 2011 on 9:36 am

  thanks jay, have send the information you asked for, thanks in advance

  Log in to Reply
  • JaySenSharma
    December 20th, 2011 on 2:38 pm

    Hi Aravindh,

    I have gone through the Thread Dumps and did not get much clue except the following:

    1). Noticed that the following object is locked: oracle/adfinternal/controller/util/RequestLock and some Threads are STUCK and trying to gain lock on this object.

    2). But i also noticed that different Threads are NOT WAITING to gain Lock on the same Instance of the oracle/adfinternal/controller/util/RequestLock Rather they are waiting fr different instance of the same class Object. Which seems to be normal. Because in general it happens. The thing could be worst if all these threads would be waiting for the same Instance of the class oracle/adfinternal/controller/util/RequestLock

    So due to point 2 I am suspecting that some processing related to ADF components are taking much longer time. I am not sure what kind of application and ADF configuration are you using but looks like this issue will require an in-depth investigation including the Application specific Configuration as well as WLS Side configuration. So it would be better if you can open a Support Case with Oracle WebLogic Team.

    At this moment just try to increase the Stuck Thread Timeout value a little longer to see if the issue still persist or not Servers —-> Configuration —-> Tuning….this is just to know whether the threads are getting enough time to process their requests or not?
    .
    .
    Keep Posting 🙂
    Thanks
    Jay SenSharma

    Log in to Reply
• Aravindh
  December 21st, 2011 on 9:23 am

  Thanks much jay for your analysis, yes i will do the same way as you suggested, thanks again

  Log in to Reply
• Aravindh
  February 1st, 2012 on 6:56 pm

  Hi,

  The following jrockit patch applies to our environment
  Oracle JRockit 28.1.x home JDK and JRE 6 Patch 13028791

  https://updates.oracle.com/Orion/PatchDetails/process_form?patch_num=13028791

  But there is no specific instruction in the readme as how we need to apply it

  when unzipped the file, there was only one directory with jrockit-jdk1.6.0_29

  So , I suppose, we have to replace the present jrockit directory

  could you please confirm whether i am doing the right thing? thanks

  Log in to Reply
• Aravindh
  February 1st, 2012 on 7:37 pm

  We had to point the app to the recent jdk and restart it , thanks

  Log in to Reply
• Aravindh
  February 23rd, 2012 on 11:16 am

  Hi Jay,

  We have EM monitoring enabled and at times we are seeing heap usage across few WLS servers – so one of my team mate tried dropping the cache and the heap usage came down

  My question is – is it recommended to do drop cache?
  what is the user impact of this?

  I suppose there will be a performance impact!!Could you please throw some light on this

  The command which was used to drop cache

  sync; echo 1 >/proc/sys/vm/drop_caches; echo 2 > /proc/sys/vm/drop_caches; echo 3 > /proc/sys/vm/drop_caches

  Log in to Reply
• pranay69
  May 31st, 2012 on 5:29 am

  Hi Jay,

  We have our application deployed in 10.3 Weblogic Application Server, and we are seeing some stuck threads when ever application is functioning. How can we increase the thread pool size for a single application in 10.3. With the work managers i guess all the applications use the threads from same thread pool. How can we increase the number of threads for a single application in Weblogic 10.3? Any specific procedure to follow. Got some info but wold like to clear out all my questions.

  Thanks,
  Pranay.

  Log in to Reply
• ramkumar4
  June 4th, 2012 on 9:48 pm

  hi my question is not belong to wlst..
  we are using oem(oracle enterprise manager). i have a doubt in this..i want to move the oms from box1 to box2 what are the step i should have to follow?
  thanks in advance…

  Log in to Reply
• pramodkulkarni.cts@gmail.com
  October 20th, 2013 on 7:41 pm

  hi I’m working on a web application. currently i’ve a requirement of making parallel web service calls in my application. To achieve this i’m planning to use workmanager in Weblogic 10.3. I’ve already made entry in weblogic.xml

  myWorkManager
  myWorkManager

  after making the changes in the weblogic.xml. I’ve made entry in my web.xml

  ….other spring mvc related servlet mappings

  wl-dispatch-policy
  myWorkManager

  Now when I restart my server i’m still getting message in the weblogic console stating default workmanager has been initiated. In other words i do not see my work manager being initiated. Also i checked the weblogic console i see workmanager which is named as default instead of myWorkManager. Also while trying to do a jndi lookup i do not see the value for the work manager.

  Can you plz let me know if i need to make any further changes to my weblogic.xml and web.xml. Also i’m using spring mvc do i need to make any changes in the spring-config.xml ??

  Log in to Reply
• inderpreet.d
  December 5th, 2017 on 11:11 am

  Hi,

  Is there a way we can enable Throttling on OSB Business Service using WLST? If yes, can you let me know how it can done.

  Log in to Reply