Anonymous_User Absent Member.
Absent Member.
270 views

IDM 4.0.x Increase in default values for JVM heap size

One thing we noticed when going from IDM 3.6.1 to 4.0.1 at a customer
was that the 4.0 remote loader had huge JVM defaults compared to the
older version.

IDM 3.6.1 (32 and 64 bit) had min = 8MB, max = 64MB
IDM 4.0.1 (64 bit only) has min = 128MB, max = 512MB

Anyone know why these were bumped so much?
Labels (1)
0 Likes
3 Replies
Anonymous_User Absent Member.
Absent Member.

Re: IDM 4.0.x Increase in default values for JVM heap size

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

64-bit JREs take more memory, at least for memory addresses and other
pointers. As a result any system near its memory limits will fail after
upgrade, so it makes sense to bump them up. Also, a bunch of issues
have come up over the years as environments increased in size where an
increase in memory would solve the problem.

Good luck.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQIcBAEBAgAGBQJQWxGfAAoJEF+XTK08PnB5ajgQALDn6mG7bFM4CGY4jfuVbMyL
WlcpSCgpI2+IuHvsugr/3FEqgWVkmLS7HwtNlDQmnWkBQBeV6Lyl8YV4+Wzu+Lsv
YrlXyhTx5Bd+iLGST/6/aES+VVvObbG5zowDTyOF7LjJ8IfxzBngb1yatHRsxkFq
P9tyo7k3MCT1ly6j4bj00zPt8zPfdvnHyYJBQCPVIJDKuBlzy2cv4XTk2Wt3q0Qv
qKaB/k2weogZVWQtWenA/FaSiacRb36dnQkg1sfDB4T5UUUHTHbvwwZlOrvZ57Zv
YKR82IOmfaf+2bu846DXpcDdd3W24Rbi65OWuwoghxGkwcVvppUkIFz+3l8WuHS6
v1XIE4Z9+IfkQgFmpRBeyKFxYSDN0JmrDbFDWbJjs1cnGJgjgXzmX7jdRJ0zpxH9
C+1e0Jcq+XTcxgZD0gGufMrvCJ7wCNEh6HurKvzmHAnUc7Y1r2kYAFblj4b++X5A
oeEjtBToQ1g8ZFnxTrX0SbOZUS5RplNloSRMutSrnzBOjPlD5153lZ8w6rRx1Wmw
D25cQRR+94EmH+Ptgvbd7nAaC3TGcM8lx6TORn3WVKV5To5TwYyqw2O0k8ahh8IQ
/NztnmOwwjDtMkyrRF9zyf2xrtc7Gck361UCreUF0Z7S2Zr4RlsAPJYu43TuxEtB
oQFmC3Zyy9DktN4AE3yV
=Tllj
-----END PGP SIGNATURE-----
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: IDM 4.0.x Increase in default values for JVM heap size

On 20.09.2012 14:52, ab wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> 64-bit JREs take more memory, at least for memory addresses and other
> pointers. As a result any system near its memory limits will fail after
> upgrade, so it makes sense to bump them up. Also, a bunch of issues
> have come up over the years as environments increased in size where an
> increase in memory would solve the problem.


At this customer, we've been running 64bit AD drivers (3.6.1) with the
default 8MB/64MB configuration for several years.

Are you saying now that this now requires 16 times as much memory (by
default?)

My understanding that the memory increase required to compensate for the
increased overhead inherent in a 64bit VM was more in the region of 50%

Is this a case of.. it's only possible to have one default setting for
creating new remote loader instances, regardless of the type of driver
shim loaded? I know from experience that moderately large XML input
files delimited text driver can easily require a max heap size of 512MB
for the remote loader.
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: IDM 4.0.x Increase in default values for JVM heap size

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> At this customer, we've been running 64bit AD drivers (3.6.1) with
> the default 8MB/64MB configuration for several years.
>
> Are you saying now that this now requires 16 times as much memory
> (by default?)


No, just that the defaults have been increased because many times the
defaults were insufficient and increasing the Xms to 128 MB was not
expected to impact many customers in a noticeably negative way.

> My understanding that the memory increase required to compensate for
> the increased overhead inherent in a 64bit VM was more in the region
> of 50%


Yes, that is my understanding as well.

> Is this a case of.. it's only possible to have one default setting
> for creating new remote loader instances, regardless of the type of
> driver shim loaded? I know from experience that moderately large XML
> input files delimited text driver can easily require a max heap size
> of 512MB for the remote loader.


Yes, as I'm sure you know the heap settings are there from the start of
the JVM and changing them based on which classes are loaded by the JVM
is not possible (would kinda defeat the purpose). These should be
configurable, though, by changing the settings for the RL:

IDM 4.0 SP2, RL Guide, Section 3.0, Configuring the Remote Loader

https://www.netiq.com/documentation/idm402/idm_remoteloader/data/be4skjr.html

The inability to predict and handle arbitrarily-large documents is a
bummer for anything Java-based... I suppose there could be ways to
mitigate that but I'm not sure how feasible it is. In the end
understanding the inputs to the system and controlling them as much as
possible is probably the best way to go to avoid needing an infinite
amount of RAM in the off chance that an application creates an
infinitely-large document, but that's not really the topic of this thread.

Good luck.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQIcBAEBAgAGBQJQXHzyAAoJEF+XTK08PnB5TGcQAMmjAAaTRb6u9M+kZL4/V3A/
hjRV38RGM5qhcAWIzHHuVRvu3rkT+pZo/irUtCk/NQ/1Gg9vYPmWzIMOiNfEj0t2
aVL34mBvDVOst+QdJwpxjvb4TSKUmCPd2HoZ5gONlGHYR8YRvcKA+fSvNLwfxTEl
YeUEAFS0J/TSU9J6s23D2xlpf197OJaDEHpR+ge1Om0vLu4kgwhbbc/ZgoKwCozN
4PRZk193MbTbZabJRrtaPTpg9yh/Pww/vSPkEHy+kPDpN/p29yYsOmBKoSJIC/8c
3v2TnDdA2h1wNoI2PGrkJ6LKLCbXKrxnqatsXa6oCIIPISVsEonCC5nzt+rfvFCS
n3gYhVxL3pHNapiqNykTnJ6MJGss2sKJxyzJCXPUTRDoj08/3/PGhCnMXsGhQla1
GUH7SIWH3bV1p/3h3bPvqOhU6k3Z8y1gwB7wj+EtDnMKX1OFZ/QRg9YXSorabAlW
5OEQjZMyiUnQASc1OGha7PPv2nYmOm91DTwBsiMdaC2ZCMuOmppAWA/rHainDLdG
maRU9GlvntSJ0MSXC9MIyx5oktrJK+IwKRRjcR1H9efCBQf9zmEK40FEd5fm9HEw
LJXTF22HFiQU9nmxNszzmScEIbtu/3eNjHD+9EquQluXlajVySdZWNs/Ia8n8JKx
+g1PXw3UtnE6Z9C0H6ST
=bV7/
-----END PGP SIGNATURE-----
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.