Anonymous_User Absent Member.
Absent Member.
648 views

Only administrative password changes sync from AD to ID Vault

IDM 4.02 engine, running on RHEL 6. AD remote loader is version 4.0.0,
running on a Windows 2008R2 domain controller. After working perfectly
for over a year, we now have an issue where passwords sync normally from
the vault to AD, but not in the other direction, unless they're set by
an administrator. As far as I can tell, password changes performed by
the user via CTL+ALT+DEL don't even show up in the driver trace. How is
that possible?

Thoughts?

Thanks

Labels (1)
0 Likes
40 Replies
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault

Nothing about the API differs with either type of password change, so the
problem is most-likely a missing filter from one of your DCs. This
happens once in a while when the MAD folks add a new DC and then forget to
tell you, and all is well except that now some passwords stop
syunchronizing mysteriously, or do not work this time, but do work on the
next attempt. A password change by an admin probably happens on a DC,
while password changes by regular users happen on workstations and then
choose a DC at random. Could be bad luck.

Anyway, if you get nothing at all in trace then the problem is that you
are missing a filter, or maybe you have one misconfigured to not send to
the current RL-running system.

--
Good luck.

If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below...
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault

ab,

>
> if you get nothing at all in trace then the problem is that you
> are missing a filter, or maybe you have one misconfigured to not send to
> the current RL-running system.
>


No DCs have been added. All filters show as "running" from the server
that's hosting the remote loader. Here's an oddity though: when I look
at the properties for the filter on one of the DCs, it lists the host
machine twice: once with just the hostname, once with the FQDN. I'll
try removing the bare hostname and restarting that DC.

Thanks



0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault

On 11/01/2013 11:50 AM, Black, Douglas wrote:
> ab,
>
>>
>> if you get nothing at all in trace then the problem is that you
>> are missing a filter, or maybe you have one misconfigured to not send to
>> the current RL-running system.
>>

>
> No DCs have been added. All filters show as "running" from the server
> that's hosting the remote loader. Here's an oddity though: when I look at
> the properties for the filter on one of the DCs, it lists the host machine
> twice: once with just the hostname, once with the FQDN. I'll try removing
> the bare hostname and restarting that DC.


I wouldn't expect duplicate entries to hurt, but it certainly will not
help. I'd remove the bare hostname too. Let us know if it helps. If
not, try changing a test user's password on each DC locally using MMC and
see if any of those passwords do not synchronize immediately. If they all
do, well, not sure then.

In case it helps, the way that the filter works is that a password is
captured using an API that microsoft provides which takes the password as
the DC receives it from whatever (workstation, local MMC, anything). That
value is then encrypted and sent along to the machine running the driver
(usually a Remote Loader machine on another DC, or maybe the same DC)
which then sends the value along to IDM (some details omitted for
brevity). As a result, the method of password change really should not
matter, and if it does then something is amiss with the API microsoft
provides because that should work the same way for all cases.

--
Good luck.

If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below...
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault

ab,
>
> I wouldn't expect duplicate entries to hurt, but it certainly will not
> help. I'd remove the bare hostname too. Let us know if it helps. If
> not, try changing a test user's password on each DC locally using MMC and
> see if any of those passwords do not synchronize immediately. If they all
> do, well, not sure then.
>



You were correct in your assessment. When I connected ADUC directly to
the domain controller that had duplicate host server entries, the
password change did not propagate to the ID vault. I then connected to
the DC that is hosting the remote loader and changed the password again,
and that change *was* processed correctly. I haven't tried any other
DCs as yet.

Removing the bare hostname from the DC where it was showing did not
resolve the issue. I have not tried rebooting the DC yet.

Our AD admin is asking if there is any way to log the PWFILTER traffic
among DCs so we know for sure what's going on. I suspect the answer is
'no', but thought I would pass his question along.


Thanks

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault

On 11/01/2013 01:57 PM, Black, Douglas wrote:
>
> You were correct in your assessment. When I connected ADUC directly to
> the domain controller that had duplicate host server entries, the password
> change did not propagate to the ID vault. I then connected to the DC that
> is hosting the remote loader and changed the password again, and that
> change *was* processed correctly. I haven't tried any other DCs as yet.
>
> Removing the bare hostname from the DC where it was showing did not
> resolve the issue. I have not tried rebooting the DC yet.
>
> Our AD admin is asking if there is any way to log the PWFILTER traffic
> among DCs so we know for sure what's going on. I suspect the answer is
> 'no', but thought I would pass his question along.


The password filter stuff used to have its own event log so you could read
it with eventvwr, but I do not know that this is still the case, and after
Password Sync the only messages I remember seeing in there were really
basic and not helpful so they are not used much for troubleshooting anymore.

Other than that is one possible option that is hard to exploit except
manually, and very easy to use in a way that breaks your password sync
setup, so be warned. This is not hard to do, but if you do it incorrectly
you will break password sync, and then die a terrible death (worse than
the one you'll experience anyway for running windows as a "server" instead
of using a server OS). Disclaimers, etc., blah blah.

First, the encrypted passwords are stored in the registry under
HKLM/Software/Novell/PwFilter/data which you can mostly access with
regedit/regedt32 but permissions are not granted to anybody, even domain
admins or the local administrator account, so you cannot see anything from
the key with the username through the values inside that represent the
password, timestamp, and sync status.

"That's easy, ab, I'll just give myself rights."

Yes, that is what everybody thinks, maybe says, and then does. This is
the cause of the death. Goodbye. The reason is that in order to give
rights you usually end up changing all permissions within here which
breaks password sync. Fixing it is easy, and requires a couple of DC
reboots, and every shortcut you try to take to save time is a waste of
time. Permissions in this world are just so terrible in these cases.

If you did not take that silly route (it's really a bad idea... do not
change rights anywhere under HKLM/Software/Novell/PwFilter ever, at all,
even once) there are other options. Because the system (as in 'SYSTEM')
has rights to this area (since it is the thing writing/reading in there)
you can read stuff inside if you have 'SYSTEM' rights. The problem with
windows... well, one of the many problems with windows is that you cannot
easily run things as 'SYSTEM'. It is possible, though, in a couple of
ways. One way is to setup a service, set that service to run as Local
System Account ('SYSTEM'), set that service to interact with the desktop,
login to the real console (or something equivalent to it... console zero
via RDP, VNC, etc.) and then run the service. That service, then, should
be loading regedit which will launch as 'SYSTEM' and be way too powerful,
but will let you see things without changing any permissions.

How do you do this? 'srvany.exe' is the tool that lets you launch
anything. Create a custom service using instsrv calling srvany, and then
define the parameters for the service so that srvany calls regedit.exe.
See Google for specific instructions as they are pretty common.

Side note: the service you create above does not end by itself when you
close regedit, so you must manually end it in order to start it again and
get another regedit instance. Quirky, but at least it works.

--
Good luck.

If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below...
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault

ab <ab@no-mx.forums.netiq.com> wrote:
>
> How do you do this? 'srvany.exe' is the tool that lets you launch
> anything. Create a custom service using instsrv calling srvany, and then
> define the parameters for the service so that srvany calls regedit.exe.
> See Google for specific instructions as they are pretty common.




--
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below...
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault

ab,
>
> The password filter stuff used to have its own event log so you could read
> it with eventvwr, but I do not know that this is still the case, and after
> Password Sync the only messages I remember seeing in there were really
> basic and not helpful so they are not used much for troubleshooting anymore.
>
> Other than that is one possible option that is hard to exploit except
> manually, and very easy to use in a way that breaks your password sync
> setup, so be warned. This is not hard to do, but if you do it incorrectly
> you will break password sync, and then die a terrible death (worse than
> the one you'll experience anyway for running windows as a "server" instead
> of using a server OS). Disclaimers, etc., blah blah.
>
> First, the encrypted passwords are stored in the registry under
> HKLM/Software/Novell/PwFilter/data which you can mostly access with
> regedit/regedt32 but permissions are not granted to anybody, even domain
> admins or the local administrator account, so you cannot see anything from
> the key with the username through the values inside that represent the
> password, timestamp, and sync status.
>
> "That's easy, ab, I'll just give myself rights."
>
> Yes, that is what everybody thinks, maybe says, and then does. This is
> the cause of the death. Goodbye. The reason is that in order to give
> rights you usually end up changing all permissions within here which
> breaks password sync. Fixing it is easy, and requires a couple of DC
> reboots, and every shortcut you try to take to save time is a waste of
> time. Permissions in this world are just so terrible in these cases.
>
> If you did not take that silly route (it's really a bad idea... do not
> change rights anywhere under HKLM/Software/Novell/PwFilter ever, at all,
> even once) there are other options. Because the system (as in 'SYSTEM')
> has rights to this area (since it is the thing writing/reading in there)
> you can read stuff inside if you have 'SYSTEM' rights. The problem with
> windows... well, one of the many problems with windows is that you cannot
> easily run things as 'SYSTEM'. It is possible, though, in a couple of
> ways. One way is to setup a service, set that service to run as Local
> System Account ('SYSTEM'), set that service to interact with the desktop,
> login to the real console (or something equivalent to it... console zero
> via RDP, VNC, etc.) and then run the service. That service, then, should
> be loading regedit which will launch as 'SYSTEM' and be way too powerful,
> but will let you see things without changing any permissions.
>
> How do you do this? 'srvany.exe' is the tool that lets you launch
> anything. Create a custom service using instsrv calling srvany, and then
> define the parameters for the service so that srvany calls regedit.exe.
> See Google for specific instructions as they are pretty common.
>



I tried several different approaches to running regedit as the local
system account, most recently this one:

http://social.technet.microsoft.com/Forums/windowsserver/en-US/98a97aee-c62b-4683-94ab-3777899cf7de/application-as-a-service-srvanyexe-in-windows-server-2008?forum=winserverMigration

That uses "sc create" instead of instsrv.exe. When I tried using
instsrv.exe as mentioned elsewhere, I just got "cannot locate file",
even if I provide a full path to both instsrv.exe and srvany.exe.

Using "sc create RegPeek binPath= C:\Windows\System32\srvany.exe
DisplayName= "Registry Peek", I am able to create a service that starts,
but the application does not. I'm still trying to figure this out.

Interestingly, I was told by a NetIQ support engineer that it's
perfectly okay to change permissions to that registry key, comments in
TID3614450 notwithstanding. I am, however, still reluctant to do so.


0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault

Black, Douglas wrote:

> I tried several different approaches to running regedit as the local
> system account, most recently this one:
>
>

http://social.technet.microsoft.com/Forums/windowsserver/en-US/98a97aee-c62b-4683-94ab-3777899cf7de/application-as-a-service-srvanyexe-in-windows-server-2008?forum=winserverMigration
>
> That uses "sc create" instead of instsrv.exe. When I tried using
> instsrv.exe as mentioned elsewhere, I just got "cannot locate file",
> even if I provide a full path to both instsrv.exe and srvany.exe.
>
> Using "sc create RegPeek binPath= C:\Windows\System32\srvany.exe
> DisplayName= "Registry Peek", I am able to create a service that
> starts, but the application does not. I'm still trying to figure this
> out.
>
> Interestingly, I was told by a NetIQ support engineer that it's
> perfectly okay to change permissions to that registry key, comments
> in TID3614450 notwithstanding. I am, however, still reluctant to do
> so.


sc is great, I use it all the time to configure services, if that works
for you - definitely use it.

This has worked for me at several customers.
http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx

psexec -i -s regedit.exe -m

The -m at the end of the regedit is optional. What it does is allows
you to run more than one copy of regedit at the same time (which is I
suspect part of your problem). Otherwise regedit steadfastly refuses to
open a second instance on the same computer (even when they run as
different users).

Whilst optional it can also be useful to allow you to compare and
contrast between the registry keys you can see as your regular admin
user and as LocalSystem.

Regarding the NetIQ support engineer's comment, I'd still err on the
side of safety and respect the TID here. There is likely a way one can
add the correct rights without impacting the existing logic - but it's
far to easy to inadvertantly screw it up.

--
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below...
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault

Alex McHugh,:

>
> This has worked for me at several customers.
> http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx
>
> psexec -i -s regedit.exe -m
>
> The -m at the end of the regedit is optional. What it does is allows
> you to run more than one copy of regedit at the same time (which is I
> suspect part of your problem). Otherwise regedit steadfastly refuses to
> open a second instance on the same computer (even when they run as
> different users).
>
> Whilst optional it can also be useful to allow you to compare and
> contrast between the registry keys you can see as your regular admin
> user and as LocalSystem.
>



That worked! Now I can get back to troubleshooting. If I get time, I'll
add a comment to the TID recommending this approach instead of creating
a service to run regedit.exe.


Thanks,


Doug




0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault

On Fri, 01 Nov 2013 19:57:21 +0000, Black, Douglas wrote:

> You were correct in your assessment. When I connected ADUC directly to
> the domain controller that had duplicate host server entries, the
> password change did not propagate to the ID vault. I then connected to
> the DC that is hosting the remote loader and changed the password again,
> and that change *was* processed correctly. I haven't tried any other
> DCs as yet.


Are the RPC ports open and working correctly? This smells like a
communications problem.


> Our AD admin is asking if there is any way to log the PWFILTER traffic
> among DCs so we know for sure what's going on. I suspect the answer is
> 'no', but thought I would pass his question along.


Sysinternal's DebugView shows many interesting things for the password
sync filter and its friends.


--
--------------------------------------------------------------------------
David Gersic dgersic_@_niu.edu
Knowledge Partner http://forums.netiq.com

Please post questions in the forums. No support provided via email.
If you find this post helpful, please click on the star below.
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault

David Gersic,
> Are the RPC ports open and working correctly? This smells like a
> communications problem.
>

....
>
> Sysinternal's DebugView shows many interesting things for the password
> sync filter and its friends.
>
>


I haven't tried DebugView yet, but the IDM log shows "SetFilterInfo()
returned 0x000006BA", indicating that the RPC server is unavailable.
This error appears for all DCs except the one running the RL.

I gave up and opened a support incident, and the engineers pointed to
that and said that I need to open ports 135, 137, 138, and 139 on all
DCs. Everything else I've read, from Microsoft and others, says that
only port 135 needs to be open. Furthermore, we most definitely did not
change the firewall settings on all six DCs recently, so I have trouble
believing that's the issue.


Thoughts?

Thanks

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault


Did you check the firewall on the DC running the RL, it could be
blocking the incoming connections.


--
Kgallogly
------------------------------------------------------------------------
Kgallogly's Profile: https://forums.netiq.com/member.php?userid=2387
View this thread: https://forums.netiq.com/showthread.php?t=49115

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault

On Thu, 07 Nov 2013 19:09:11 +0000, Black, Douglas wrote:

> I gave up and opened a support incident, and the engineers pointed to
> that and said that I need to open ports 135, 137, 138, and 139 on all
> DCs.


I'd tend to believe NTS on this. But IIRC, there's a list of ports in the
MAD Driver documentation, so you should be able to reference that.


--
--------------------------------------------------------------------------
David Gersic dgersic_@_niu.edu
Knowledge Partner http://forums.netiq.com

Please post questions in the forums. No support provided via email.
If you find this post helpful, please click on the star below.
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Only administrative password changes sync from AD to ID Vault

David Gersic,
>
>> I gave up and opened a support incident, and the engineers pointed to
>> that and said that I need to open ports 135, 137, 138, and 139 on all
>> DCs.

>
> I'd tend to believe NTS on this. But IIRC, there's a list of ports in the
> MAD Driver documentation, so you should be able to reference that.
>


The driver documentation says that port 135 must be open in all cases,
but ports 137, 138 and 139 need to be open if you are running NetBIOS
over TCP. I'm trying to find out if we are.


0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.