Narrowing down OutOfMemory error


UA 4.0.2 patch D on Linux x64
I know E is out but customer is afraid of patching...

I'm troubleshooting an OOM error and I have enabled memory dumps on
OutOfMemory

errors.

The customer has a max heap size of 1.8GB according to recommendations
by NetIQ

support.

Looking through those dump I can see a large number of objects that take
up most

of the heap space when the OOM error occurs.

com.novell.srvprv.impl.vdata.model.VirtualEntity
com.novell.srvprv.impl.vdata.model.VirtualEntityAttribute

The VirtualEntity objects contain the VirtualEntityAttribute objects.

For example there are 8 398 VirtualEntity objects and 66 752

VirtualEntityAttribute objects that take over 1,2GB of heap.

The VirtualEntityAttribute objects that take up most of the space
contain

AssignedResources, InheritedRoles, MemberOf.

I'm trying to identify what's causing it to list *all* role/resource
memberships

for all those users.

Any ideas?

Would looking at a role/resource assignement tab in the role catalog
cause to

retrieve *all* values for a User?

Would using any of the built-in reports in UA cause this?

Can I put a limit on the query so that it doesn't try to list so many
users?

I have two stack traces here:

http://paste.opensuse.org/872f3169


http://paste.opensuse.org/76a8eabb


--
alekz
------------------------------------------------------------------------
alekz's Profile: https://forums.netiq.com/member.php?userid=974
View this thread: https://forums.netiq.com/showthread.php?t=52124

Parents
  • Which web application service is used? x86_32 or x86_64?

    --
    Good luck.

    If you find this post helpful and are logged into the web interface,
    show your appreciation and click on the star below...
  • The JBoss that comes with IDM 4.0.2 and 64-bit Java 7.

    On 2014-11-06 23:00, ab wrote:
    > Which web application service is used? x86_32 or x86_64?
    >

  • Do you have the hs_err*.log file from the OOM? Is it a plain old OOM, or
    do you also happen to have another message mentioning PermGen (a problem
    with the permanent generation specifically) or GCOverheadException (a
    setting in the garbage collector that throws the OOM error in certain
    cases when garbage collection is not recovering much memory for the time
    taken)?

    Any ulimit settings apply to JBoss's java? Post the output of the
    following, substituting the PID with the proper java process PID:


    cat /proc/PID/limits


    --
    Good luck.

    If you find this post helpful and are logged into the web interface,
    show your appreciation and click on the star below...
  • On 11/07/2014 08:00 AM, ab wrote:
    > Do you have the hs_err*.log file from the OOM? Is it a plain old OOM, or
    > do you also happen to have another message mentioning PermGen (a problem
    > with the permanent generation specifically) or GCOverheadException (a
    > setting in the garbage collector that throws the OOM error in certain
    > cases when garbage collection is not recovering much memory for the time
    > taken)?
    >
    > Any ulimit settings apply to JBoss's java? Post the output of the
    > following, substituting the PID with the proper java process PID:
    >
    >

    > cat /proc/PID/limits
    >

    >

    Greetings,
    I have seen this before from a few customers who have workflows that
    will return all users in a certain role via a DAL Query. Some of these
    Roles have many thousands of users assigned.



    --

    Sincerely,
    Steven Williams
    Lead Software Engineer
    NetIQ
  • On 11/07/2014 09:35 PM, Steven Williams wrote:
    > On 11/07/2014 08:00 AM, ab wrote:
    >> Do you have the hs_err*.log file from the OOM? Is it a plain old OOM, or
    >> do you also happen to have another message mentioning PermGen (a problem
    >> with the permanent generation specifically) or GCOverheadException (a
    >> setting in the garbage collector that throws the OOM error in certain
    >> cases when garbage collection is not recovering much memory for the time
    >> taken)?
    >>
    >> Any ulimit settings apply to JBoss's java? Post the output of the
    >> following, substituting the PID with the proper java process PID:
    >>
    >>

    >> cat /proc/PID/limits
    >>

    >>

    > Greetings,
    > I have seen this before from a few customers who have workflows that
    > will return all users in a certain role via a DAL Query. Some of these
    > Roles have many thousands of users assigned.
    >
    >
    >

    Greetings,
    If the case above is what you are seeing then you will need to limit
    the results on the Entity in the DAL.

    But I would suggest that you track what is going on to better
    confirm what is going on. Also, apply the latest Public Patch for 4.0.2

    --

    Sincerely,
    Steven Williams
    Lead Software Engineer
    NetIQ
  • Hi ab,
    I can check the ulimit setting in a couple of days...

    No hs_err files, I think they only get created when the JVM crashes
    which is not the case here.

    On 2014-11-07 14:00, ab wrote:
    > Do you have the hs_err*.log file from the OOM? Is it a plain old OOM, or
    > do you also happen to have another message mentioning PermGen (a problem
    > with the permanent generation specifically) or GCOverheadException (a
    > setting in the garbage collector that throws the OOM error in certain
    > cases when garbage collection is not recovering much memory for the time
    > taken)?
    >
    > Any ulimit settings apply to JBoss's java? Post the output of the
    > following, substituting the PID with the proper java process PID:
    >
    >

    > cat /proc/PID/limits
    >

    >

  • Thanks Steven,
    I will start to go through the PRD's looking for getRole* calls, to try
    to identify the cause.

    Do you know if it's possible from a memory dump to see maybe the logged
    in users at the time, or to see the PRD that called the getRole API?



    On 2014-11-08 03:41, Steven Williams wrote:
    > On 11/07/2014 09:35 PM, Steven Williams wrote:
    >> On 11/07/2014 08:00 AM, ab wrote:
    >>> Do you have the hs_err*.log file from the OOM? Is it a plain old
    >>> OOM, or
    >>> do you also happen to have another message mentioning PermGen (a problem
    >>> with the permanent generation specifically) or GCOverheadException (a
    >>> setting in the garbage collector that throws the OOM error in certain
    >>> cases when garbage collection is not recovering much memory for the time
    >>> taken)?
    >>>
    >>> Any ulimit settings apply to JBoss's java? Post the output of the
    >>> following, substituting the PID with the proper java process PID:
    >>>
    >>>

    >>> cat /proc/PID/limits
    >>>

    >>>

    >> Greetings,
    >> I have seen this before from a few customers who have workflows that
    >> will return all users in a certain role via a DAL Query. Some of these
    >> Roles have many thousands of users assigned.
    >>
    >>
    >>

    > Greetings,
    > If the case above is what you are seeing then you will need to limit
    > the results on the Entity in the DAL.
    >
    > But I would suggest that you track what is going on to better confirm
    > what is going on. Also, apply the latest Public Patch for 4.0.2
    >

Reply
  • Thanks Steven,
    I will start to go through the PRD's looking for getRole* calls, to try
    to identify the cause.

    Do you know if it's possible from a memory dump to see maybe the logged
    in users at the time, or to see the PRD that called the getRole API?



    On 2014-11-08 03:41, Steven Williams wrote:
    > On 11/07/2014 09:35 PM, Steven Williams wrote:
    >> On 11/07/2014 08:00 AM, ab wrote:
    >>> Do you have the hs_err*.log file from the OOM? Is it a plain old
    >>> OOM, or
    >>> do you also happen to have another message mentioning PermGen (a problem
    >>> with the permanent generation specifically) or GCOverheadException (a
    >>> setting in the garbage collector that throws the OOM error in certain
    >>> cases when garbage collection is not recovering much memory for the time
    >>> taken)?
    >>>
    >>> Any ulimit settings apply to JBoss's java? Post the output of the
    >>> following, substituting the PID with the proper java process PID:
    >>>
    >>>

    >>> cat /proc/PID/limits
    >>>

    >>>

    >> Greetings,
    >> I have seen this before from a few customers who have workflows that
    >> will return all users in a certain role via a DAL Query. Some of these
    >> Roles have many thousands of users assigned.
    >>
    >>
    >>

    > Greetings,
    > If the case above is what you are seeing then you will need to limit
    > the results on the Entity in the DAL.
    >
    > But I would suggest that you track what is going on to better confirm
    > what is going on. Also, apply the latest Public Patch for 4.0.2
    >

Children
No Data