We are having problem to grant certificates on OMi.
After deploy an agent version 12.00, We change some agent configuration as below.
ovconfchg -ns eaagt -set OPC_NODENAME <SERVER_NAME>
ovconfchg -ns eaagt -set OPC_IP_ADDRESS <SERVER_IP_ADDRESS>
ovconfchg -ns bbc.cb -set SERVER_BIND_ADDR <SERVER_IP_ADDRESS>
ovconfchg -ns bbc.http -set SERVER_BIND_ADDR <SERVER_IP_ADDRESS>
ovconfchg -ns bbc.http -set CLIENT_BIND_ADDR <SERVER_IP_ADDRESS>
after this, We send another ovcert -certreq, and on OMi Certificate Request page the certificate appears, but after Accept, the status stay in "Granted" and never changes.
We tested route, icmp, name resolution and agent configuration, but any server that I try to certificate this happens.
Have Anyone already seen that?
I tried unninstall and install again, deleted the node first on OMi, but still happening the same.
We called the HP support that analyzed and suggested to reinstall all the environment because some problem found probably in the future could impact in other problems.
But after reinstall we faced other certificate problems, so them after many tests I could understand how exactly works in OMi 10 in a distributed environment with multihomed ip address devices.
When you install without configure any custom settings, the agent send the certificate to Gateway using the main interface.
when you grant in console, isn't the gateway that send the certificate back, the DPS that send the certificate back, so your agent needs to communicate with you DPS too.
Our environment doesn't use the main interface (production) to monitor the devices, so we need to set some configurations in the agent to all communication use the administration interface (secondary interface).
After the installation We configured the parameters below and resent the ovcert command.
ovconfchg -ns eaagt -set OPC_IP_ADDRESS <AGENT_IP_ADDRESS>
ovconfchg -ns bbc.cb -set SERVER_BIND_ADDR <AGENT_IP_ADDRESS>
Hope you are doing great.
Based on the behavior you are sharing this looks expected behavior , when a certificate request arrives to OMi it will show up as pending, then once it is granted it will not change to any other state again.
If what you mean is that even if it says is granted in the OMi console the agent is not able to communicate with OMi then it could be that something is blocking the connection in this direction "OMI > OA" if the connection is blocked in this direction then the Operations Agent will never know that the Management server "OMi" has granted the certificate.
In other words, the reply back from OMi to the agent never arrives and hence the agent is not able to communicate with the OMI server.
You can check this:
From OMI to the Operations Agent
ping <fqdn of the agent>
bbcutil –ping <FQDN of the agent>
Is it a fresh agent installation ? or an upgrade from a previous version?
I await to the outputs to see how the communication looks like.
thanks for the fast reply,
We don't have firewall in this network and firewall are disabled on both OS (Agent and OMi).
I tested the commands that you asked, and ping is OK, but bbcutil ping is not, but I don't no why!
C:\Users\<sensured>>ping <censured>.<censured> Pinging <censured>.<censured> [<censured>] with 32 bytes of data: Reply from <censured>: bytes=32 time<1ms TTL=127 Reply from <censured>: bytes=32 time<1ms TTL=127 Reply from <censured>: bytes=32 time<1ms TTL=127 Reply from <censured>: bytes=32 time<1ms TTL=127 Ping statistics for <censured>: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Average = 0ms C:\Users\<sensured>>bbcutil -ping <censured>.<censured> <censured>.<censured>: (bbc-288) status=eServiceError coreID= bbcV= appN= appV= conn=0 time=416 ms
I checked the agent and all seems good.
C:\Users\<censured>>ovc hpsensor HP Compute Sensor AGENT,OA (9388) Executando oacore Operations Agent Core AGENT,OA (6188) Executando opcacta OVO Action Agent AGENT,EA (11492) Executando opcmsga OVO Message Agent AGENT,EA (3172) Executando ovbbccb OV Communication Broker CORE (11532) Executando ovcd OV Control CORE (7296) Executando ovconfd OV Config and Deploy COREXT (9260) Executando
telnet on 383 port are working in both sides.
These two servers (OMi and agent) have 2 or more IPs address, but we are using the IPs that are in the same network, and the routes are OK, i tested and checked that the communication are between the same network.
I checked the status using ovc command on the OMi gateway and DPS server and all are Running.
I did a research to grant using command line, but I'm receiving an error too.
C:\windows\system32>ovcm -listpending -l RequestID: b09b5692-0d34-7583-13e9-b3f2fde099e0 Context: CN: b237a002-0666-7583-0faf-839c1b134954 Nodename: <censured>.<censured> IPAddress: <censured> PeerAddress: <censured> Platform: Windows 6.1, CPU: x64, OSBITS: 64 InstallType: Auto TimeReceived: 10/30/2015 2:34:52 PM E. South America Daylight Time C:\windows\system32>ovcm -grant b09b5692-0d34-7583-13e9-b3f2fde099e0 ERROR: (sec.cm.client-55) Call to server failed. SoapFaultException: faultcode: Server faultstring: The RPC server encountered an unexpected OvXplIo::IOException_t. faultactor: https://localhost:383/com.hp.ov.sec.cm.certificateserver/ certreqhandler/ faultdetail: <detail> <bbce:bbcdetail xmlns:bbce="http://openview.hp.com/xmlns/bbc/soap/faultdetail/ 1.0"> <bbce:exception>OvXplIo::IOException_t</bbce:exception> <bbce:message>(xpl-68) End of input data reached.</bbce:message> </bbce:bbcdetail> </detail>
Thanks for th prompt response Diego,
Did you restart the communication broker process at the Operations Agent side after doing the configuration changes ?
This agent command needs to be run after IPs change.
Also please provide the output of this command from the OA managed node:
At the OMI side (DPS) in case of Distributed environment please run this command and provide the output
bbcutil -ping https://localhost:383/com.hp.ov.sec.cm.certificateserver/
Also these outputs from the OMi GTW
ovcoreid -ovrg server
From the operations agent side please provide these outputs
Follow the results:
On agent side:
C:\Users>ovbbccb -reinit ERRO: (bbc-292) O Intermediário de comunicações do OV no host 'localhost' negou a solicitação devido a uma falha de autorização. Assegure-se de que os certificados de SSL adequados estejam instalados e configurados. C:\Users>ovc hpsensor HP Compute Sensor AGENT,OA (9388) Executando oacore Operations Agent Core AGENT,OA (6188) Executando opcacta OVO Action Agent AGENT,EA (11492) Executando opcmsga OVO Message Agent AGENT,EA (3172) Executando ovbbccb OV Communication Broker CORE (11532) Executando ovcd OV Control CORE (7296) Executando ovconfd OV Config and Deploy COREXT (9260) Executando
On DPS side:
C:\>bbcutil -ping https://localhost:383/com.hp.ov.sec.cm.certificate?server/ https://localhost:383/com.hp.ov.sec.cm.certificate?server/: (bbc-289) status=eServiceUnknown time=515 ms
On GTW side:
C:\>ovcoreid -ovrg server aafe4e6c-5dbe-4cf8-851a-5442b66d35e9
On the agent side again:
C:\>ovconfget sec.core.auth MANAGER=<censored> (Gateway FQDN) MANAGER_ID=aafe4e6c-5dbe-4cf8-851a-5442b66d35e9 C:\>ovconfget sec.cm.client CERTIFICATE_SERVER=<censored> (Gateway FQDN) C:\>bbcutil -getcbport localhost:383
C:\>bbcutil -reg NOTA: Enviando consulta ao Intermediário de comunicações do OV no caminho: 'http://localhost:383/' BasePath=/Hewlett-Packard/OpenView/Coda/ Protocol=HTTPS BindAddress=::1 Port=51109 Authentication=NONE PID=6188 BasePath=/com.hp.ov.conf.core/bbcrpcserver/ Protocol=HTTPS BindAddress=::1 Port=51058 Authentication=REMOTE PID=9260 BasePath=/com.hp.ov.conf.core/checkpolicy/bbcrpcserver/ Protocol=HTTPS BindAddress=::1 Port=51058 Authentication=REMOTE PID=9260 BasePath=/com.hp.ov.ctrl.ovcd/ Protocol=HTTPS BindAddress=::1 Port=50577 Authentication=REMOTE PID=7296 BasePath=/com.hp.ov.depl/bbcfxserver/ Protocol=HTTPS BindAddress=::1 Port=51058 Authentication=ALL PID=9260 BasePath=/com.hp.ov.depl/bbcrpcserver/ Protocol=HTTPS BindAddress=::1 Port=51058 Authentication=ALL PID=9260 BasePath=/com.hp.ov.eaagt.actr/ Protocol=HTTPS BindAddress=::1 Port=51073 Authentication=ALL PID=11492 BasePath=/com.hp.ov.eaagt.msga.hbp/ Protocol=HTTPS BindAddress=::1 Port=51079 Authentication=NONE PID=3172 BasePath=/com.hp.ov.health/oacore/ Protocol=HTTPS BindAddress=::1 Port=51096 Authentication=NONE PID=6188 BasePath=/com.hp.ov.health/opcacta/ Protocol=HTTPS BindAddress=::1 Port=51077 Authentication=NONE PID=11492 BasePath=/com.hp.ov.health/opcmsga/ Protocol=HTTPS BindAddress=::1 Port=51086 Authentication=NONE PID=3172 BasePath=/com.hp.ov.sec.cm.certificateclient/msg/ Protocol=HTTPS BindAddress=::1 Port=50561 Authentication=NONE PID=7296 BasePath=/com.hp.ov.sec.cm.certificateclient/rpc1/ Protocol=HTTPS BindAddress=::1 Port=50561 Authentication=ALL PID=7296 BasePath=/com.hp.ov.sec.cm.certificateclient/rpc2/ Protocol=HTTPS BindAddress=::1 Port=50561 Authentication=REMOTE PID=7296 BasePath=/hpcs/ Protocol=HTTPS BindAddress=::1 Port=51089 Authentication=NONE PID=9388 BasePath=/oacore.oacore.rrListener/ Protocol=HTTPS BindAddress=::1 Port=51093 Authentication=NONE PID=6188 BasePath=/oacore.oacore.rrListener/raw/ Protocol=HTTPS BindAddress=::1 Port=51097 Authentication=NONE PID=6188 BasePath=/oacore.oacore/ Protocol=HTTPS BindAddress=::1 Port=51116 Authentication=NONE PID=6188 BasePath=/oacore.oacore/bbcrpcserver/ Protocol=HTTPS BindAddress=::1 Port=51116 Authentication=NONE PID=6188
have you installed OMi 10.01 IP2? If yes then please be aware of this:
If you install Operations Agent 12.00.078 or higher after installing
OMI_00120(OMi 10.01 IP2) you need to update the HPOvSecCS LCore package.
Please do so by running the following command from the command line:
msiexec /i %TOPAZ_HOME%\installation\HPOMi1001IP204\lcore\HPOvSecCS-12.00.078-Win5.2_64-release.msi /qn
Is your environment distributed (GTW and DPS separate) or everything is running on the same machine ?
If your environment has DPS and GTW installed in different machines then you need to update the below parameter on the ovconfget of the Operations Agent side must be the DPS FQDN
CERTIFICATE_SERVER=<censored> (Gateway FQDN)
You can open the ovconfget with the next command:
Scroll down and change the FQDN
After that restart the Agent
Instead of running ovcert -certreq it is nicer and more effective to activate the agent as follows:
cscript oainstall.vbs -c -a -activateonly -srv <GTW FQDN> -cert_srv <DPS FQDN>
Then check the certificates request console at the OMi interface. If the request was not automatically granted then go ahead and grant it.
That's not correct: you never use the DPS server as the certificate server.
You either use the gateway or the load balancer.
If the Load Balancer doesn’t allow non-SSL requests to come through pick a gateway instead (but you lose high availability)
The agent is not supposed to connect directly to the DPS.
According to documentation and our HP Archtictec, this new version 10.01 the certificate server is the DPS, but GW is able to certificate too.
This is the informantion we asked, because we are implemmenting the solution here.
Thanks very much!
I hope you had a nice weekend,
I tried what you asked to do, but still the same, the certificate is not geting in the agent.
I think we have some comunication problem but I don't know what to search.
I see that when I try to test the command bbcutil -ping from the managed nod to Gateway, I'm able to communicate, and a telnet test on 383 port we got successful too.
C:\temp\Agent_OMI_1200_WIN>bbcutil -ping <gateway server> <gateway server>: status=eServiceOK coreID=f2a7a442-47a7-757f-0354-aa2b356fa93e bbcV=12.00.078 appN=ovbbccb appV=12.00.078 conn=5 time=226 ms
But when i use bbcutil -ping from gateway server to managed node I got an error, but telnet on por 383 we got successful.
D:\HPBSM\installation>bbcutil -ping <censored>.<censored> <censored>.<censored>: (bbc-288) status=eServiceError coreID= bbcV= appN= appV= conn=0 time=287 ms
In managed node i testes bbcutil for itself and got sucessful.
C:\temp\Agent_OMI_1200_WIN>bbcutil -ping <censored>.<censored> <censored>.<censored>: status=eServiceOK coreID=b237a002-0666-7583-0faf-839c1b134954 bbcV=12.00.078 appN=ovbbccb appV=12.00.078 conn=1 time=225 ms
Do you know which port is needed to communicate with the agent? other tests that I could do?
I tested the port, route, configuration that I know only.
can you please tell me which documentation you refer to? I will then enter a defect.
Technically the certificate server runs on the DPS, however, the agent must use the load balancer or the GW. Otherwise some features will not work correctly (e.g. HA).
Only the OMi gateways/DPS set the DPS as certificate server.