GC Allocation Failure messages in Connector agent.wrapper.out log file

[cyberie]

Hello,

I'm seeing GC allocation failure messages continuously in my Windows connector logs (agent.wrapper.out) These messages appears like almost every second. However, i do not see any issue in pulling the logs or any strange behaviour on the event channel but im afraid of anything which is being lost, can anyone explain me if this is normal behaviour.

Thanks,

Raj

[Marijo Mandic]

Hello,

1) I assume you see these two types of events in:
<connector_home>/current/logs/agent.out.wrapper.log

2) Types:
a) [root@centos logs]# cat agent.out.wrapper.log | grep "GC (Allocation Failure)"
INFO   | jvm 1    | 2018/07/02 21:10:36 | [GC (Allocation Failure)  176478K->71665K(249088K), 0.0191078 secs]
INFO   | jvm 1    | 2018/07/02 21:17:36 | [GC (Allocation Failure)  176625K->72085K(249088K), 0.0198958 secs]
INFO   | jvm 1    | 2018/07/02 21:24:36 | [GC (Allocation Failure)  177045K->72522K(249088K), 0.0228252 secs]

b)  [root@centos logs]# cat agent.out.wrapper.log | grep Full
INFO   | jvm 1    | 2018/07/03 13:04:15 | [Full GC (Metadata GC Threshold)  83112K->12916K(249088K), 0.2479668 secs]
INFO   | jvm 1    | 2018/07/03 13:04:39 | [Full GC (Metadata GC Threshold)  52072K->24615K(249088K), 0.5524438 secs]
INFO   | jvm 1    | 2018/07/03 14:38:34 | [Full GC (Metadata GC Threshold)  83011K->12664K(249088K), 0.3090687 secs]

3) If you have "Full GC" too often, like for example few in a minute then you can try to increase SmartConnector JVM Memory Heap size to make it "breathe" better (restart of SmartConnector is needed to apply settings, take backup of file):
<connector_home>/current/user/agent/agent.wrapper.conf

a) Default lines that need change:
# Initial Java Heap Size (in MB)
wrapper.java.initmemory=256

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=256

b) Example of increase to 512 MB (be sure that you have enough free RAM):
# Initial Java Heap Size (in MB)
wrapper.java.initmemory=512

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=512

Regards,

Marijo

[cyberie]

Hi Marijo,

Thanks for looking into it and for your help.

No I do not see Full GC messages only GC Failure messages appears continuously one after other almost every second, I'm attaching a snapshot of the logs. 

Kind regards,

Raj

[Marijo Mandic · Accepted solution]

Hello Raj,

1) Would not consider this an issue.
2) You can try to increase the JVM size to see if the amount of messages is lower.
3) SmartConnector will continue to work as it is, does not seem a problem as:
a) there is no Cache
b) ET and HT are up
c) EPS and T are almost the same, so what goes in it goes out (matches with C=0)
4) This seems that is WUC SmartConnector, so it is retrieving the logs from Windows host and then sending to ESM, so no event loss should happen.

Regards,

Marijo

[cyberie]

Got it, Thanks Marijo,

Cheers!!!!

[Sachin]

Hi,

I was facing the same error, the connector was showing ET and HT=Down so from the above chat I got the idea that it is related to the destination I just checked reachability and reregistered the destination after that it got resolved.

Thanks, everyone.

[manojs]

Per my conversation with MF support recently, GC Allocation Failure messages are quite frequent in new connector framework 7+.

They suggested to ignore it as connector is working fine unless we see Yellow/Red Zone errors and frequent Full GC`s.