jeschaff
New Member.
1947 views

noob question, if anyone is still using analyzer


As a first time user of Analyzer 4.5, I could uses some advice / insight
from anyone who has experiance using the current or older versions of
Anlyzer and is willing to take the time to answer a few noob qustions
that weren't addressed in the 'NetIQ 4.5 documentation'
(http://tinyurl.com/zj3obma).

Thank you in advance for your time and or your responses.... it is
much appreciated!!!
-Jeff S.


HERE IS WHAT I'M ATTEMPTING TO DO:

I'm attempting to import user data from three different
eDirectory / LDAP trees, and then compare the data so that I can
consolidate the eDirectory instances with the aggregate data. For that
effort I'm looking at between 45,000 and 50,000 user records per tree...
with 47 different attributes each. The attributes consist of half stock
and half custom, including some multi valued attributes such as member
and security equals. Similarly I would like to compare the groups in
each eDirectory tree, rouglhy 2500 groups per tree with 5 attributes
each (including multi valued attributes like membership and equivalent
to me). For this effort I've been successful in installing and setting
up Analyzer 4.5. I've been successful in importing my users to the
default database (it took just under 13 hours per user dataset). I've
also been successful in passing the uniqueness test, and have even
started my matching efforts for two user datasets... using three
attributes in the mathing rule (cn, wokforceid, and uuid). However 18
hours in and it is still running with no indicator of progress, or how
much longer it might take. Thus I have some questions:

*Question 1)* Is what I'm attempting to do realistic and or feasible?

*Question 2)* What things if any can I do to optimize performance of
the dataset imports and or dataset comparisions?

*Question 3)* Can / Does Analyzer 4.5 support comparing every value of
a multi valued attribute... when comparing datasets?

*Question 4)* The context in one of my trees has different case for the
O, but other wise case matches between the three trees. Will this cause
me a problem with group membership, security equals, equivalent to me,
and ACL matching?

*Question 5)* For comparing groups, or OUs across different trees...
is there any advice you can offer for matching? I imported my groups in
about an hour (roughly 2500 per tree). But my uniqueness comparison
fails because I can't seem to specify DN, or anything related to the
context that the group exists in.... thus I'm having difficulties
getting my uniqueness test to pass as using the CN or any other
combination of attributes which would be consistent across trees comes
back with less then a 90% uniqueness rate.


For the groups dataset import I'm specifying 5 ObjectClasses:
groupOfNames, ldapGroup. nestedGroupAux, posixGroup, Top
And I'm interested in comparing roughly 7 attributes: cn, member,
groupMember, groupMembership, equivalentToMe, gidNumber, description

For the OrganizationlUnit dataset import I'm specifying one ObjectClass:
organizationalUnit
And I'm interested in comparing the attributes: cn, description, ACLs
(trustee rights), associated password policies, & intruder detection
settings/configuration.

*Question 6)* I saw in one of the old forum posts that using a MySQL
database might yield significantly better performance. In the post it
referred me to the Analyzer documentation, and the configure preferences
section where I would specify a database using MySQL. However I don't
know anything about MySQL... so what would I have to do to set that up.
The info seems to be missing from the Analyzer documentation. More
specifically if I change the config preference in Analyzer does it
create and populate a MySQL database by default? Or do I first have to
create such a database, with a schema and or tables predefined, then
drop it on my workstation, and finally point analyzer 4.5 to it?

*Question 7)* Are there any gotchas I should be aware of... I think I
read in the forums that values longer then 255 characters might be a
problem.... I think only my group memberships related attributes might
possibly exceed this limitation if that is the only gotcha.


--
jeschaff
------------------------------------------------------------------------
jeschaff's Profile: https://forums.netiq.com/member.php?userid=400
View this thread: https://forums.netiq.com/showthread.php?t=56511

Labels (1)
0 Likes
14 Replies
Knowledge Partner
Knowledge Partner

Re: noob question, if anyone is still using analyzer

jeschaff <jeschaff@no-mx.forums.microfocus.com> wrote:
>
>
>


Are you using the default embedded database? It has limitations and
possibly the performance can be better that way.

>

much longer it might take. Thus I have some questions:
>
> *Question 1)* Is what I'm attempting to do realistic and or feasible?
>


It can be a slow tool.

> *Question 2)* What things if any can I do to optimize performance of

the dataset imports and or dataset comparisions?
>


Try a different database. I've not played with specific indexing options
but that might also be a big help.

> *Question 3)* Can / Does Analyzer 4.5 support comparing every value of

a multi valued attribute... when comparing datasets?
>


Good question. Ages since I played with it. Recall really struggling with
multiple values. However I was mostly working with delimited text imports
and there were other limitations there.

> *Question 4)* The context in one of my trees has different case for the

O, but other wise case matches between the three trees. Will this cause
me a problem with group membership, security equals, equivalent to me,
and ACL matching?
>


Yes with default database IIRC. You need to use another DB

> *Question 5)* For comparing groups, or OUs across different trees...

is there any advice you can offer for matching? I imported my groups in
about an hour (roughly 2500 per tree). But my uniqueness comparison
fails because I can't seem to specify DN, or anything related to the
context that the group exists in.... thus I'm having difficulties
getting my uniqueness test to pass as using the CN or any other
combination of attributes which would be consistent across trees comes
back with less then a 90% uniqueness rate.
>
>


DN almost always varies slightly from tree to tree. I would match on
different attributes instead.


>
> *Question 6)* I saw in one of the old forum posts that using a MySQL

database might yield significantly better performance.

Definitely. I believe I also used one of the free/basic MS SQL database
products also.

Had this set up at my prior job so I don't have the config to conveniently
refer to.

Essentially it uses JDBC to talk to the database. You need to specify the
libraries that support your database and the connection parameters.

You might also get a DBA to help you with this bit.




> More specifically if I change the config preference in Analyzer does it

create and populate a MySQL database by default? Or do I first have to
create such a database, with a schema and or tables predefined, then
drop it on my workstation, and finally point analyzer 4.5 to it?
>


The former if I recall correctly.

> *Question 7)* Are there any gotchas I should be aware of... I think I

read in the forums that values longer then 255 characters might be a
problem.... I think only my group memberships related attributes might
possibly exceed this limitation if that is the only gotcha.

There are always gotchas.

--
If you find this post helpful and are logged into the web interface, show
your appreciation and click on the star below...
Alex McHugh - Knowledge Partner - Stavanger, Norway
Who are the Knowledge Partners
If you appreciate my comments, please click the Like button.
If I have resolved your issue, please click the Accept as Solution button.
0 Likes
jeschaff
New Member.

Re: noob question, if anyone is still using analyzer


Thanks for the quick response Alex.

-Jeff S.


--
jeschaff
------------------------------------------------------------------------
jeschaff's Profile: https://forums.netiq.com/member.php?userid=400
View this thread: https://forums.netiq.com/showthread.php?t=56511

0 Likes
Knowledge Partner
Knowledge Partner

Re: noob question, if anyone is still using analyzer

jeschaff wrote:

> I'm attempting to import user data from three different
> eDirectory / LDAP trees, and then compare the data so that I can
> consolidate the eDirectory instances with the aggregate data.


Just a thought: it might be easier to set up a fresh "consolidation" Edirectory
tree and use IDM to import/match/merge the objects from all three existing
trees into it. You'd have all the power and performance of IDM and could sync
the result back into the source trees easily once you are satisfied with the
results...

--
http://www.is4it.de/en/solution/identity-access-management/

(If you find this post helpful, please click on the star below.)
______________________________________________
https://www.is4it.de/identity-access-management
0 Likes
Knowledge Partner
Knowledge Partner

Re: noob question, if anyone is still using analyzer

Lothar Haeger <lothar.haeger@is4it.de> wrote:
> jeschaff wrote:
>
>> I'm attempting to import user data from three different
>> eDirectory / LDAP trees, and then compare the data so that I can
>> consolidate the eDirectory instances with the aggregate data.

>
> Just a thought: it might be easier to set up a fresh "consolidation" Edirectory
> tree and use IDM to import/match/merge the objects from all three existing
> trees into it. You'd have all the power and performance of IDM and could sync
> the result back into the source trees easily once you are satisfied with the
> results...
>


When you have a hammer (IDM), everything looks like a nail (driver).

Good point to go with something you are more familiar with that can also do
the job. As especially with delimited text as input I have found Analyzer
sorely lacking.



--
If you find this post helpful and are logged into the web interface, show
your appreciation and click on the star below...
Alex McHugh - Knowledge Partner - Stavanger, Norway
Who are the Knowledge Partners
If you appreciate my comments, please click the Like button.
If I have resolved your issue, please click the Accept as Solution button.
0 Likes
Knowledge Partner
Knowledge Partner

Re: noob question, if anyone is still using analyzer

Alex Mchugh wrote:

> so that I can
> >> consolidate the eDirectory instances with the aggregate data.

> >
> > You... could sync the result back into the source trees easily once you
> > are satisfied with the results...
> >

>
> When you have a hammer (IDM), everything looks like a nail (driver).


I rather though, Analyzer seems not the best tool to hammer the consolidated
data back into three source trees. 🙂

--
http://www.is4it.de/en/solution/identity-access-management/

(If you find this post helpful, please click on the star below.)
______________________________________________
https://www.is4it.de/identity-access-management
0 Likes
jeschaff
New Member.

Re: noob question, if anyone is still using analyzer


Well it isn't so much moving the consolidated data back into three
trees.

The actual goal is that we have a synchronized Identity Vault tree (IDV)
and an Authorization (Auth) tree. We were trying to validate first
that they are in fact in perfect sync as we have observed some
discrepancies.

Additionally we have a third eDirectory tree (Legacy), which is
currently connected to our IDV via an eDir to eDir driver but doesn't
sync everything... and that Legacy tree is currently handling some
integration with web sites, and other stuff. So once the IDV and Auth
tree sync was verified we then wanted to Compare the legacy tree's users
and their security attributes (custom attributes and group memberships)
to the IDV. This is to make sure that all the data currently being
consumed from the Legacy tree is also in the synced IDV and thus Auth
trees. Once we confirm this we can move those integration to our Auth
tree, and decommission / retire that third eDirectory tree (Legacy).
So that was the actual end goal we were trying to accomplish. In this
one might ask why we don't just export what we have in our Legacy tree,
and import it into the IDV/Auth tree to get everything in that set of
directories... or push it all via IDM. Well as I said only some of
the data is synced between the IDV and the Legacy tree.. and while the
IDV tree connects to both our Auth and Legacy tree via IDM the sync
isn't by directional... and we've gotten into a state were both trees
have similar data in some cases but depending on the integration the
source of authority could be one tree vs the other... so we are trying
to be very careful in consolidating the data such that we don't break
any existing integration. Basically it's a bad state that we are
trying to correct.

Anyhow I first played around with sed, awk, grep, and ice (or ldap
search) and found that with some effort, for single valued attributes...
I could export all users from each tree with the one attribute I was
interested in, turn it into a delimited file, sort it, and then diff the
resulting files from each tree to see where discrepancies were which
would in turn allow me to make corrections to the IDV and Auth trees
where necessary. However with multi valued attributes I had less
success with this type of approach (not being much of a unix / linux
guy... or scripting guru for that matter). My problem there was while I
could get the exported users data with the group memberships into a
delimited file, I could not get the order within that record sorted such
that the resulting delimited files would match up enou that a diff
between the three different outputs was useful / meaningful.

From there I looked for tools on the internet that might make this
easier / more automated... rather then working through lots of exports
and text comparisons. Through those efforts I was unsuccessful in
finding any good tools I could make work, which would read exported ldap
(ldiff formatted data), that would then properly order and compare the
data to tell me which accounts were orphaned and which accounts just had
missing data when comparing each of the three trees. And that is where
I was when I found the NetIQ Analyzer tool that is packaged with IDM...
which from it's description / documentation sounded like it might help
me in reaching my end goal... by comparing data and minimizing the time
spent exporting individual ldap objects and then comparing them against
similar exports from two other trees.

So I'm providing the full context of my dilemma in case it changes
anyone's suggestions as it relates to the suggested approach and or
tools. Its also worth mentioning that while I have an existing IDM
setup, I'm by no means an expert on the drivers. I can troubleshoot
existing drivers, maintain them pretty well, make small modifications to
them, and setup some very basic drivers such as eDir to eDir or LDAP...
but I'm no expert in coding / setting up IDM drivers and or performing
any sort of complicated java / ecma scripting.

-Jeff S.


--
jeschaff
------------------------------------------------------------------------
jeschaff's Profile: https://forums.netiq.com/member.php?userid=400
View this thread: https://forums.netiq.com/showthread.php?t=56511

0 Likes
Knowledge Partner
Knowledge Partner

Re: noob question, if anyone is still using analyzer


I use Analyzer and I like it!
"Internal" database is pretty slow, but I use "external" MySQL for
database.

Analyzer is pretty good for initial analyze of different systems. I used
for data comparison from ITIM/eDir/AD.


--
If you find this post helpful, please show your appreciation by clicking
on the star below :cool:
------------------------------------------------------------------------
al_b's Profile: https://forums.netiq.com/member.php?userid=209
View this thread: https://forums.netiq.com/showthread.php?t=56511

0 Likes
jeschaff
New Member.

Re: noob question, if anyone is still using analyzer


al_b;273223 Wrote:
> I use Analyzer and I like it!
> "Internal" database is pretty slow, but I use "external" MySQL for
> database.
>
> Analyzer is pretty good for initial analyze of different systems. I used
> for data comparison from ITIM/eDir/AD.


Al_b.... thanks for the response. I would agree Importing data to the
local database is very slow, along with performing large comparisons
between imported data sets. That said once you get beyond the slowness
it does seem to work pretty good if you can define unique patterns for
comparisons. My only complain is that I can't seem to use the full DN
in my uniqueness tests. I also find the data analysis functionality for
individual attributes to be very powerful, such as testing for
consistent formats and correcting when it doesn't match. Though I'm
still trying to self learn how to do more advance stuff with this
functionality and the java script that can be used to enhance it.

That said I have two questions if you would be so kind as to indulge
me.

1) If using MySQL is significantly faster... is there any write ups or
documentation that you could point me at that would explain to a noob
how to integrate Analyzer with MySQL. I'd now say I'm pretty familiar
with Analyzer but I know very little about MySQL... Such as can I run
it locally on my laptop, do I need to have a full blown server, when I
create the database to I need to prepopulate the schema/tables some how,
etc...

2) I think I read somewhere, possibly in this forum, that there were
some know issues / caveats if you switched from a local database to
MySQL. Thus I was curious if you had such an experience and if so
could you share what those this are so that I can look out for them / be
aware of them.

Thanks,
-Jeff S.


--
jeschaff
------------------------------------------------------------------------
jeschaff's Profile: https://forums.netiq.com/member.php?userid=400
View this thread: https://forums.netiq.com/showthread.php?t=56511

0 Likes
Knowledge Partner
Knowledge Partner

Re: noob question, if anyone is still using analyzer


jeschaff;273225 Wrote:
> Al_b.... thanks for the response. I would agree Importing data to the
> local database is very slow, along with performing large comparisons
> between imported data sets. That said once you get beyond the slowness
> it does seem to work pretty good if you can define unique patterns for
> comparisons. My only complain is that I can't seem to use the full DN
> in my uniqueness tests. I also find the data analysis functionality for
> individual attributes to be very powerful, such as testing for
> consistent formats and correcting when it doesn't match. Though I'm
> still trying to self learn how to do more advance stuff with this
> functionality and the java script that can be used to enhance it.
>
> That said I have two questions if you would be so kind as to indulge
> me.
>
> 1) If using MySQL is significantly faster... is there any write ups or
> documentation that you could point me at that would explain to a noob
> how to integrate Analyzer with MySQL. I'd now say I'm pretty familiar
> with Analyzer but I know very little about MySQL... Such as can I run
> it locally on my laptop, do I need to have a full blown server, when I
> create the database to I need to prepopulate the schema/tables some how,
> etc...
>
> 2) I think I read somewhere, possibly in this forum, that there were
> some know issues / caveats if you switched from a local database to
> MySQL. Thus I was curious if you had such an experience and if so
> could you share what those this are so that I can look out for them / be
> aware of them.
>
> Thanks,
> -Jeff S.


I did it number of years ago and I didn't remember any "official"
documentation.
It is looks like time to repeat and document all these steps.
I think, that I will do it and will put it here or on Cool Solution
site.


--
If you find this post helpful, please show your appreciation by clicking
on the star below :cool:
------------------------------------------------------------------------
al_b's Profile: https://forums.netiq.com/member.php?userid=209
View this thread: https://forums.netiq.com/showthread.php?t=56511

0 Likes
Knowledge Partner
Knowledge Partner

Re: noob question, if anyone is still using analyzer


I submitted "How-to configure NetIQ Analyzer for IDM to use external
MySQL database." to Cool Solution site.
I hope that it will be available soon...


--
If you find this post helpful, please show your appreciation by clicking
on the star below :cool:
------------------------------------------------------------------------
al_b's Profile: https://forums.netiq.com/member.php?userid=209
View this thread: https://forums.netiq.com/showthread.php?t=56511

0 Likes
Knowledge Partner
Knowledge Partner

Re: noob question, if anyone is still using analyzer

After pretty long waiting period my article How to configure NetIQ Analyzer for IDM to use external MySQL database finally published.
https://www.netiq.com/communities/cool-solutions/configure-netiq-analyzer-idm-use-external-mysql-database/

Happy New Year for everybody!
0 Likes
Knowledge Partner
Knowledge Partner

Re: noob question, if anyone is still using analyzer

On 12/30/2016 10:56 PM, al b wrote:
>
> After pretty long waiting period my article How to configure NetIQ
> Analyzer for IDM to use external MySQL database finally published.
> https://www.netiq.com/communities/cool-solutions/configure-netiq-analyzer-idm-use-external-mysql-database/
>
> Happy New Year for everybody!


Good article, good reading, well done.

Now do more such articles!


0 Likes
Knowledge Partner
Knowledge Partner

Re: noob question, if anyone is still using analyzer

Thank you very much, Geoffrey!
Your comments are very valuable to me!

I think, that I will create more CS articles. 🙂
0 Likes
Knowledge Partner
Knowledge Partner

Re: noob question, if anyone is still using analyzer

On 1/12/2017 10:26 AM, al b wrote:
>
> Thank you very much, Geoffrey!
> Your comments are very valuable to me!
>
> I think, that I will create more CS articles. 🙂


Hey, they pay in Amazon gift certs. Amazon Canada really stinks,
compared to Amazon US, but you know how it is. (It is really
surprisingly different, since I shop between the two occasionally).

But it is a good way to get your name out there into the field.


0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.