Anonymous_User Absent Member.
Absent Member.
494 views

Best MG Membership Criteria and Service Map Load Times


Can someone explain in some detail what the difference in performance
would be (if any) in using Filtered Event Views based on the Server View
generated from different Membership Rules in a Management Group for me
pelase?

For instance:
Using an MG Membership Property of Repository "View" of Master for each
QDB versus using a Rule such as "AppManager Installed" with
"Repositories Greater than 8.0.0.0" or versus just using a simple static
"List" inclusion.

Questions:
Is one type of Membership more dynamic (based on a SQL View or QDB
Master View) and need to be parsed more often than a simple Rule-based
inclusion?
How often does the Membership get refreshed?
Is there a limit to the number of Filtered Event Views placed onto a
Service Map before seeing degradation in performance of Svc Map load
times?
What are some other 'tweaks' that can be done to improve load times for
Service Maps and avoid timeouts? (config timeout edited to allow up to 3
minutes already) BTW - We do not have embedded service maps in use
either.
Does applying Permissions to an MG (with all servers) also affect
Service Map load times also?
Another observation is that we have other service maps which will load
relatively quickly but they are built with Filtered Event Views on a
much smaller subset of Server Views (other MGs)

Problem:
Need to have multiple Filtered Event Views (~16) that apply to ALL
Servers (~500) on multiple QDBs and we are experiencing quite a deal of
latency on loading of Service Map and sometimes if fails to load
altogether. All of these Filtered Event Views have only a few "Server
Filters" and none of them are based on Server. Most "Server Filters"
are "Severity Less Than Or Equal To 20" and Knowledge Script "Is Like
NT_Service*". Since our Operations Staff needs to watch all Servers for
certain conditions like Disk, Hardware, Down Servers, etc. we have to
have Event View reflect all Servers.


--
armstrongge
------------------------------------------------------------------------
armstrongge's Profile: https://forums.netiq.com/member.php?userid=4253
View this thread: https://forums.netiq.com/showthread.php?t=51454

0 Likes
2 Replies
Anonymous_User Absent Member.
Absent Member.

Re: Best MG Membership Criteria and Service Map Load Times


All Management Groups are based on QDB Views (or Server Groups within
QDB Views).... which may sound incorrect, but is the case. Either
directly based on a View (like Master, NT or SQL). Or in the case of a
Rule, the Rule is associated with a "hidden" view in the QDB and this is
evalued every minute in the QDB (via the "Uphold Rule Based Dynamic
View" SQL Job.

So a MG based directly on a View has the least impact in the QDB since
it is not evaluated. It just gets refreshed when the membership of the
view changes - which usually would be when a discovery happens.

Rule Based Dynamic Views are basically the result of executing the SQL
which is generated by a rule you define. Some of these can get quite
complicated and so take some time to evaluate. The more of these you
have, the more work - for the QDB. So I guess the answer to your first
question is that Rule Based is more costly as these are all evaluated
every minute whereas the View (or Server Group) based are not
evaluated... some other activity such as discovery refreshes the
membership.

Lists are basically the same as Rule Based - though the SQL is more
straightforward, they are still hidden views in the QDB.

Filtered views in the CCDB are also SQL which is evaluated. But
evaluated (generally) only when the UI "fetches" the view. In essence,
all items in a view (like servers, events or jobs) are stored in a base
table (ie ComputerServer for servers). When you look at a view in the CC
UI, a procedure is executed that works out which items are associated
with the MG that you are working with. At that time, any server filters
are processed as part of fetching those items.

So the workload would then depend on how many UIs are fetching those
filtered views.

BUT ... the CQS also processes these. The CQS runs the sync commands
(those commands that fetch data from the QDBs). But it also has a number
of threads which process the information already in the QDB. If you run
this query:-

SELECT * FROM SyncTable WHERE DataSourceId IS NULL

then you will see those commands... you will also see how long each one
takes to complete (LastExecutionTime is the time in seconds the
"previous" run took to complete... the "current" run might be in
progress when you look at the table). The commands that have the biggest
effect are:-

EXEC dbo.CalculateObjectSeverity
EXEC dbo.CalculateObjectSeverityForFilteredViewsAndSMVs

Depending on the configuration of SQL, the 2nd command may have multiple
instances...

In 8.2 with the latest HF in place, you have the option of disabling
those TreeView flashing icons. Doing this will also remove some of those
CCDB processing threads. This can be useful in terms of performance
because it means that the CQS then stops processing EVERY view in the
CCDB... So that becomes a trade-off... But you can (with 8.2 and the
latest HF) get a balance as you can enable/disable for all. Or you can
enable only some of those views - so it may not be necessary to have the
flashing icons for all the views in all the MGs, but maybe one or 2 are
important... in which case, disable all but those you require. The
effect is that the CQS then has less background processing to do which
reduces the load on the CCDB.

To enable/disable the CQS calculation - in the UI, go to Options and
look at "Event Severity Status". If unchecked then no calculation is
done and so no flashing icons anywhere. If enabled the calculation is
done, and you can also then decide which views will participate. By
default all will, but you can look at the properties of individual views
and on the "General" tab the check box for "Calculate Event Severity for
this view".

So for "best" performance and a balance of "best" usablity, you would
globally enable the calculation - but disable for most views. Enabling
only those views which are important. These options are not available in
earlier versions, so you need 8.2 with the most recent patches.

In terms of the actual filters - as with rules, these are basically
turned into SQL which is appended to the SQL used to retrieve the items
you are looking at. The more complex a filter, the more complex the SQL
that produces. But in the most part that is "fine". However, the SQL to
avoid if possible is where the condition is "In Last X ... " - just
because the resulting SQL can be less efficient to execute and calculate
than other conditions.

In general terms - whilst a Rule is still SQL which gets evaluated, it
is evaluated "in the background" and away from the CCDB. If you can use
a rule to generate a smaller subset of servers for a Management Group
before you then use filters in the CCDB for views in that MG - this is
better for CCDB (and CC UI) performance that having a MG which has lots
of servers in it that you then filter with server filters on one or more
views in that MG.

Hope this helps....


--
Andy Doran
Software Engineer Consultant (NetIQ)
------------------------------------------------------------------------
andy_doran's Profile: https://forums.netiq.com/member.php?userid=3937
View this thread: https://forums.netiq.com/showthread.php?t=51454

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Best MG Membership Criteria and Service Map Load Times


Andy,

Thank you for that information Andy... that helps me tremendously sir!
I think I have identified a culprit in slowness with map load but would
like your feedback/thoughts on though. In migrating from AM7 to AM8 we
duplicated all of the Filtered Event Views exactly as they appeared in
AM8 (minus server names and other items that didn't exist initially). I
have identified three Filtered Event Views that were filtering for KS
Names that did not get 'ported' to AM8 deployment and consequently
simply did not exist. So I removed these altogether and speeds
dramatically improved. These three Filtered Event Views were operating
against ALL Servers on multiple repositiories... as well as a handful of
other FEVs like Disk, Services, and other OS-related items are which are
having no difficulties at all. Would non-existant KS Names in the FEVs
cause such extreme performance degradation like this in your
experiences?


--
armstrongge
------------------------------------------------------------------------
armstrongge's Profile: https://forums.netiq.com/member.php?userid=4253
View this thread: https://forums.netiq.com/showthread.php?t=51454

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.