Many times we come across NodeManager is not reachable issue and its one of the conman issues which lot of people face it, in many cases we have found that this issue happens due to missing step of nmEnroll. It means you might have not Enrolled your NodeManager with the Admin Server and thus the NodeManager is not reachable. Hence we have come up with this post so that anyone can resolve the same issue if the below steps are been followed properly:
.
.
Steps to Resolve NodeManager is not reachable issue
Step1) Start the AdminServer using startScript “startWebLogic.sh/cmd”
Step2) Start the NodeManager.
Step3) Login to admin-console and then see if the NodeManager is Reachable or not at the below console path
Console Path:
Machines -> <YOUR_MACHINE_NAME> -> Monitoring (tab) -> Node Manager Status (sub-tab) -> Status: Reachable
If its NOT REACHABLE then please do the following from the Admin Server Box to Enroll the NodeManager
a) Open a command prompt and run "setWLSEnv.sh/cmd" in it then run the following commands java weblogic.WLST connect('weblogic','weblogic','t3://localhost:7001') nmEnroll('C:/bea103/user_projects/domains/7001_Domain','C:/bea103/wlserver_10.3/common/nodemanager') b)Then again check in the console path if NodeManager is Reachable.
Step4). Once NodeManager is Enrlolled and Reachable in the admin console then continue the following steps.
Step5). Kill the AdminServer because now we need to start it using WLST command nmStart() and NOT using the start scripts.
Step6). Open a command prompt run “setWLSEnv.sh/cmd” and then run the following command
java weblogic.WLST
Step7). Now connect to the NodeManager using the following command (7001_Domain is the Domain Name for example.)
nmConnect('weblogic','weblogic','localhost','5556','7001_Domain')
Step8). Once you are successfully connected to the Nodemanager please start the AdminServer using the following command
nmStart('AdminServer')
Step9). Once the Server is started run the following command to get the AdminServer Status
nmServerStatus('AdminServer')
Now if you can see the Admin Server status that means now your AdminServer is now reachable. Hope these steps solves your NodeManager is not reachable.
Regards,
Ravish Mody
March 1st, 2011 on 11:18 am
HI Team,
I have an issue with the nodemanager configuration in weblogic 11g. Configured node manager and configured server startup params in the weblogic console. Nodemanager status on the console is reachable. I have managed instances on different machines and all the nodemanagers are reachable. But not able to start managed server instances. Can you please help:
I am seeing the following warning in the logs:
weblogic.nodemanager.common.ConfigException: The domain ‘LTOSB’ at ‘/u01/app/oracle/admin/domains’ was not registered in the nodemanager.domains file and dynamic domain registration is not supported. Please register the domain in the nodemanager.domains file.
at weblogic.nodemanager.server.NMServer.getDomainManager(NMServer.java:291)
at weblogic.nodemanager.server.Handler.handleDomain(Handler.java:227)
at weblogic.nodemanager.server.Handler.handleCommand(Handler.java:109)
at weblogic.nodemanager.server.Handler.run(Handler.java:71)
at java.lang.Thread.run(Thread.java:619)
nodemanager.domains file has the entry for the above mentioned domain
Thanks for your help in advance..
March 1st, 2011 on 11:02 pm
Hi weblogic_user,
try to remove/rename the “nodemanager” directory present inside “$WL_HOMEcommon” directory and then once again enroll the NodeManager to the AdminServer then try to start your Server again.
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
March 1st, 2011 on 11:23 pm
Hi Jay,
Thanks for your reply.
I resolved the issue, it was a configuration error on my part. As I said, all nodemanagers were reachable but I was not able to start the manage server.
In Server start parameters for each managed server I had wrong domain path
Managed Server -> Server Start -> root directory ( i had this setting wrong).
After fixing this i was able to start the manage server without any issue
Thank you
March 7th, 2011 on 1:37 pm
Hi Team,
I have an issue with the node manager and SSL.
Background:
WebLogic 11g has been installed on host dev1.host.com and OSB domains have been configured. All the servers were managed through nodemanager. No issues.
Recently, dev1.host.com has been recabled and re IP’ed to prod1.host.com
Now I configured another domain(servers/machines) which uses new IP’s and hostnames (weblogic 11g has not be reinstalled) and started node manager without any issues but when I login to the admin console nodemanager is not reachable.
Stacks trace from nodemanager startup log:
javax.net.ssl.SSLKeyException: [Security:090482]BAD_CERTIFICATE alert was received from prod1.host.com – 172.xx.xx.xx. Check the peer to determine why it rejected the certificate chain (trusted CA configuration, hostname verification). SSL debug tracing may be required to determine the exact reason the certificate was rejected.
at com.certicom.tls.interfaceimpl.TLSConnectionImpl.fireException(Unknown Source)
at com.certicom.tls.interfaceimpl.TLSConnectionImpl.fireAlertReceived(Unknown Source)
at com.certicom.tls.record.alert.AlertHandler.handle(Unknown Source)
Admin Server log:
Demo certs are been used in production mode. I guess, Since we have re IP’ed the linux boxes we might be seeing this errors.
Can you please let me know how we can troubleshoot this issue.
March 7th, 2011 on 1:39 pm
FYI., Stack from Admin Server log:
Thank you in advance…
March 7th, 2011 on 1:44 pm
Sorry.. I was not able to upload the admin server log stack:
<Certificate Chain received from prod1.host.com – 172.xx.xx.xx failed hostname verification check. Certificate contained dev1.host.com but Check expected prod1.host.com
Thanks
March 10th, 2011 on 3:29 pm
Hi weblogic_user ,
We will have to turn of host name verification and the basic validation check of the certificates.
We can do it by specifying the following flag in startWeblogic.sh
-Dweblogic.security.SSL.ignoreHostnameVerification=true -Dweblogic.security.SSL.enforceConstraints=off
And the following flag in startNodeManager.sh
-Dweblogic.nodemanager.sslHostNameVerificationEnabled=false
-Dweblogic.security.SSL.enforceConstraints=off
thanks,
sandeep
April 8th, 2011 on 10:15 pm
Hello Ravish…
I have worked with wlst in windows environment, it works fine.
But the weblogic.WLST class in the weblogic jar file in the $WL_HOME/server/lib does not work in my case. Can you help me figuring out what is the problem. Even I tried this
./selWLSEnv.sh
java -jar $WL_HOME/server/lib/weblogic.jar weblogic.WLST
it always says config file missing in the domain directory.
I tried to get the server status by doing this:
./setWLSEnv.sh
java weblogic.Admin -url -username -password GETSTATE -pretty AdminServer
it does not give me anything…just exists printing nothing.
April 8th, 2011 on 10:43 pm
Hi supratim,
Try opening a fresh new shell prompt then go to the WL_HOME servers bin folder and set your environment using two (.) dots something link this
bin]$ . ./setWLSEnv.sh
Then try to run the below command
bin]$ java weblogic.WLST
Regards,
Ravish Mody
April 8th, 2011 on 10:50 pm
Hi Supratim,
The command which you are using to start WLST is Wrong thats why it is saying “Do u want to create config.xml”….
Reason:
Suppose if you have an Executable Jar with name xyz.jar then to execute this jar you need to run the following command:
java -jar xyz.jar
Mistakenly you are trying to execute “weblogic.jar” by typing the below command:
java -jar $WL_HOME/server/lib/weblogic.jar weblogic.WLST
I can understand that you want to start Weblogic WLST utility and Not the WebLogic Server itself…so in that case you need to run the following command:
java -classpath $WL_HOME/server/lib/weblogic.jar weblogic.WLST
I just replaced the “-jar” option of your command with “-classpath”
Ravish is very correct …because of you want to set the classpath then running “. ./setWLSEnv.sh” is the best utility provided by WebLogic to avoid many classpath related issues.
.
Similarly the correct Syntax to use the weblogic.Admin utility is as following:
java weblogic.Admin -username weblogic -password weblogic -url localhost:7001 GET -type ServerDebug -pretty
.
Keep Posting 🙂
Thanks
Jay SenSharma
April 24th, 2011 on 2:31 am
if I want to remotely state nodemnaeger from a differnt box of weblogic how do I use the nmenroll command?
Do I provide the domain path of the server which I want to remotely manage or do I provide the domain path of server which I want to use to manage the remote server.
Similar for the nodemanager path which path should i use? of the server which I want to remotely manage or do I provide the nodemanager path of server which I want to use to manage the remote server?
April 24th, 2011 on 3:05 am
Hi testab,
James Bayer’s (Oracle principle Product Manager) Article on NodeManager will help you in finding all the answers :
http://blogs.oracle.com/jamesbayer/2010/01/weblogic_nodemanager_quick_sta.html
Above is one of the best article on NodeManager available on “W W W”…. nothing better you can get.
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
April 24th, 2011 on 7:21 am
Thanks
August 21st, 2011 on 5:52 pm
Hi Ravish,
I am not able to start node manager using startNodemanager.sh with weblogic 10.3.4.
It was working normally in the past. but I just noe I tried and I showed the below exception.
java.lang.NullPointerException
at java.util.Hashtable.containsKey(Hashtable.java:314)
at weblogic.nodemanager.server.NMServerConfig.initNetworkInfoList(NMServerConfig.java:495)
at weblogic.nodemanager.server.NMServerConfig.getNetworkInfoList(NMServerConfig.java:485)
at weblogic.nodemanager.server.NMServerConfig.print(NMServerConfig.java:603)
at weblogic.nodemanager.server.NMServerConfig.print(NMServerConfig.java:565)
at weblogic.nodemanager.server.NMServer.(NMServer.java:166)
at weblogic.nodemanager.server.NMServer.main(NMServer.java:390)
at weblogic.NodeManager.main(NodeManager.java:31)
PLEASE SUGGEST!
Thanks in advance
Vidhangpt
August 21st, 2011 on 5:53 pm
Also find the NodeManager.properties:
#Sun Jul 17 09:57:52 BST 2011
DomainsFile=C:\Oracle\MIDDLE~1\WLSERV~1.3\common\NODEMA~1\nodemanager.domains
LogLimit=0
PropertiesVersion=10.3
DomainsDirRemoteSharingEnabled=false
javaHome=C:\Oracle\MIDDLE~1\JROCKI~1.1-3
AuthenticationEnabled=true
NodeManagerHome=C:\Oracle\MIDDLE~1\WLSERV~1.3\common\NODEMA~1
JavaHome=C:\Oracle\MIDDLE~1\JROCKI~1.1-3\jre
LogLevel=INFO
DomainsFileEnabled=true
StartScriptName=startWebLogic.cmd
ListenAddress=localhost
NativeVersionEnabled=true
ListenPort=6000
LogToStderr=true
SecureListener=true
LogCount=1
DomainRegistrationEnabled=false
StopScriptEnabled=true
QuitEnabled=false
LogAppend=true
StateCheckInterval=500
CrashRecoveryEnabled=false
StartScriptEnabled=false
LogFile=C:\Oracle\MIDDLE~1\WLSERV~1.3\common\NODEMA~1\nodemanager.log
LogFormatter=weblogic.nodemanager.server.LogFormatter
ListenBacklog=50
August 23rd, 2011 on 9:56 pm
Hi Vidhangpt,
You nodemanager.properties file entries looks to be fine. Can you let us know what all changes had you made before starting the nodemanager when this issue occurred as you had mentioned that it was working just fine earlier, try undo all the changes you have made and then try to start the nodemanager again and if still that does not work then send us the “nodemanager.log” file to contact@middlewaremagic.com
Regards,
Ravish Mody
October 19th, 2011 on 11:24 pm
Hey Ravish,
I’m new to nodemanager and there is lot of information available on the site which confuses me about implementation. Do you have any blog for configuring nodemanger in a setp by step process ?
Any kind of help is appreciated.
Thanks,
Gopal
October 19th, 2011 on 11:48 pm
Hi Gopal,
Please refer to one of the best article of “René van Wijk” (Oracle ACE) on using NodeManager along with the pack/unpack command usages for “Setting-up a WebLogic Cluster that spans Multiple Machines”
http://middlewaremagic.com/weblogic/?p=6637
Apart from above link you can refer to “James Bayers” (VMware Sr.Product Manager) blog “WebLogic NodeManager Quick Start”
http://blogs.oracle.com/jamesbayer/entry/weblogic_nodemanager_quick_sta
.
.
Keep Posting 🙂
Thanks
Jay SenSharma
October 18th, 2012 on 7:44 pm
I am having an issue hope you can help me out.
In one of the Unix boxes there are four domains configures with different versions of
weblogic. So obviously there will be multiple node managers in the same box. My point
here is how to know the exact node manager start script file for that particular
domain.
i am in confusion in finding that.
Is there any command or file or steps to find out the exact node manager file for the
exact domain.
Regards,
Chaitanya.
Hi,
I am having an issue hope you can help me out.
In one of the Unix boxes there are four domains configures with different versions of weblogic. So obviously there will be multiple node managers in the same box. My point here is how to know the exact node manager start script file for that particular domain.
i am in confusion in finding that.
Is there any command or file or steps to find out the exact node manager file for the exact domain.
Regards,
Chaitanya.
October 19th, 2012 on 1:04 pm
How did you create the nodemanager home. Note that in the file startNodemanager.[cmd or sh] (located in the ${WL_HOME}/server/bin directory)
you can set the option NODEMGR_HOME. By default, this is pointing to the ${WL_HOME}/common/nodemanager directory. If you did not change this
parameter the configuration files for the nodemanager will be present in that directory.
Take a look in the nodemanager.domains directory to see which domains are registered with the nodemanager. If you did not change anything
all four domains are registered to the same nodemanager.
October 22nd, 2012 on 10:58 am
I checked the nodemanager.domains file and only that particular domain is registered with that node manager.
Here the node manager home is in one path and the start script of the node manager is in another path. If they are in same path there wont be any confusion. so how to find the exact script file for starting node manager. And in startNodemanager.sh file is pointing to the default only.
October 22nd, 2012 on 12:57 pm
If I am understanding you correctly, you did not change anything, when the domain was created – you also did not edit the nodemanager home in the startNodemanager.sh file.
Than you can use the nodemanager.sh file to start the nodemanager. On first start it will create a nodemanager.properties file in the nodemanager home directory
(also where the nodemanager.domains is located). By default the nodemanager will listen on post 5556.
Some more information about the nodemanager set-up can be found here: http://middlewaremagic.com/weblogic/?p=7795. A step-by-step explanation of the set-up
is provided in material for a WebLogic course which is provided here: http://middlewaremagic.com/weblogic/?p=8075
October 22nd, 2012 on 3:04 pm
for your clear understanding let me give you a example.
domain home: /ops/instance/domains/
node manager home: /app/platform/weblogic/10.3.5/wlserver/common/nodemanager
&
/app/csp/instance/nodemanager
node manager log: /ops/logs//nodemanager/nodemanager.log
node manager start script file path:
/app/platform/weblogic/10.3.5/wlserver/server/bin/startNodeManager.sh
So here in node manager home there are 2 nodemanager.properties files. So while running the startNodeManager.sh file which properties file its taking and how to know that i am running the correct startNodeManager.sh file. this is my doubt.
Please help me in finding this.
Regards,
Chaitanya
October 23rd, 2012 on 12:16 pm
The startNodemanager script will look in the specified nodemanager home if there exists a nodemanager.properties file.
If does not exist it will create one. How you managed to get two nodemanager.properties files in the nodemanager home directory
is a riddle to me (you probably copied it from somewhere).
Better is to delete all the files in the nodemanager home directory (except for the nodemanager.domains file) and let the startNodemanager script
create the necessary files.
December 15th, 2012 on 12:44 am
I am getting the below error in my logs and the server fails.
< ===== FULLTHREAD DUMP ===============
This is happening in our production environment and I need to fix this urgently.
Environment details:
———————-
Weblogic: 10.3.4
Platform: Windows
PLEASE HELP….
December 15th, 2012 on 8:20 pm
Put your logs in sourcecode tags, for example,
[/sourcecode]
December 21st, 2012 on 12:33 am
December 21st, 2012 on 2:42 pm
“Critical Subsystem ServerMigration has failed. Setting server state to FAILED.Reason: ServerWLS_SOA1 failed to renew lease in the database”
Probably the database is not up, or can not be reached.
October 1st, 2015 on 7:37 pm
Hi iam having the fallowing exception when i am trying to connect admin server through WLST.please verify this once.
connect(‘weblogic’,’weblogic012′,’t3://192.168.1.152:7001′)
Traceback (innermost last):
File “”, line 1, in ?
File “”, line 22, in connect
File “”, line 648, in raiseWLSTException
WLSTException: Error occured while performing connect : Error getting the initial context. There is no server running at t3://192.168.1.152:7001
Use dumpStack() to view the full stacktrace
October 1st, 2015 on 11:12 pm
Hello Nagasuresh,
This error can be because any of the following reason:
Cause-1). Either your WebLogic Admin Server is not listening to the Address “192.168.1.152” and Port “7001”
Steps to Troubleshoot:
a). Check the Admin Server log “$DOMAIN_HOME/server/AdminServer/logs/AdminServer.log” (Admin Server name may be different in your case). In this log see if it is having that IP Address mentioned in it? Check 3-4 lines above the following message in your log “The server started in RUNNING mode”
b). Check the “$DOMAIN_HOME/config/config.xml” file on your domain controller (AdminServer box) to see if it has the Admin Server listen address & port properly defined.
Cause-2). Or you have some firewall issue which is not allowing you to connect to WebLogic via the mentioned host and port.
Steps to Troubleshoot:
a). Check if from the remote machine you are able to access the host and port? You can use the following telnet comment to see if the host and port is accessible remotely or not?
telnet 192.168.1.152 7001
b). Check if you are able to access the Admin Console using the following URL : http://192.168.1.152:7001/console
Regards
Jay SenSharma