Highlighted
Knowledge Partner
Knowledge Partner

Re: G-Suite / Google Apps Driver Crashing on Group Creation

Jump to solution
Great to know.
Found the move problem.
If a user is moved in G-Suite and shortly after changed from IDM the user gets moved back.
Wait until next day and it works fine.
Not sure this is possible to fix. Its more of how google work.
0 Likes
Highlighted
Knowledge Partner
Knowledge Partner

Re: G-Suite / Google Apps Driver Crashing on Group Creation

Jump to solution

Per the developer, it seems like the query to Google they do at the start of each event, is returning stale, old unmoved data.  And that there is some delay on the Google side. 

He is looking at fixing it, unclear if he will succeed, since if Google still has old data, how do you handle it?

0 Likes
Highlighted
Super Contributor.
Super Contributor.

Re: G-Suite / Google Apps Driver Crashing on Group Creation

Jump to solution

Thank you, @geoffc!

Sorry, but with #2, are you suggesting that I need to clean it up by, say, building-in a delay?

Thanks again.

0 Likes
Highlighted
Knowledge Partner
Knowledge Partner

Re: G-Suite / Google Apps Driver Crashing on Group Creation

Jump to solution
I guess the problem here is to know if it has moved or been renamed.
Modifies if those account would benifit of being delayed. But to keep tack of that might be a challange.
I doubt a 30sec delay is sufficient, more like 2 hours. So it has to be a list of process later or similar.

But thats just me guessing.
0 Likes
Highlighted
Contributor.
Contributor.

Re: G-Suite / Google Apps Driver Crashing on Group Creation

Jump to solution

Hello all,

Firstly, thanks go out to Geoff for sharing the updates I was giving on that list. 

There are two core items in play for this thread I'd like to address. 

 

1) The NPE and Fatal shutdown on some group member operations

Sometime in mid to late November 2019, Google made a change to their Admin API service that caused them to start throwing 404 errors seemingly at random when attempting to retrieve a group for modification. This would be problematic enough, but for some reason the errors weren't well formed as per the API. Instead of throwing JSON error objects it was sending back an HTTP 404 error with a HTML page for browser consumption. This caused their own class libraries to choke and return a kind of poison pill error object instance which, ultimately, led to the fatal error and driver shutdown. 

I still don't know the root cause of the 404 errors. Sometimes we get them on a group, but subsequent (and previous) fetches of that group have and will succeed without the 404. The query appears to be well formed and in compliance with the API. 

What I ended up doing was going through every response handler and hardening it to handle any oddity in the response object from the API service and Google's libraries. Since the group 404 errors occurred on groups that were correct and present, I made the choice to set a status RETRY on them so the event would not be lost. You can react to that differently in policy if you so choose. 

 

2) The stale data problem. 

As you may be aware, user modify, rename, and move are, from Google's perspective, all modifications on the user object. The name is primary email address, the container is org unit path, and so on. To match the IDM event model, the user handler was structured such that moves were handled via IDM <move> command docs, as were renames, and all other user attribute changes were handled through the <modify> command. 

There is a stale data problem in the Google API servers. If you have multiple commands on the same user in rapid succession, modify/modify, modify/move, modify/rename, etc, then there is a risk that the second command will receive a non-current copy of the user object. 

Each command handler fetches a fresh instance of the user, adjusts it to match the command, then sends it back to the API service as an update. Unfortunately, there are cases where the copy fetched at the start of the second (or subsequent) command is stale, this can lead to some of the changes done in the first command being overwritten or reset. 

It's difficult to reliably replicate, but I've been able to replicate it in the GAM command-line tool as well as in the G-Suite driver. 

Version 4.2.0.x has a few modifications in the move and rename pathways to try and get a more fresh copy of the object, but it's not working in all cases. 

I've got some ideas and a plan to try and better detect the stale version case, but until this is worked through, it'd be advisable to space out multiple commands on the same user object. I don't think you need large time delays, personally. I've seen good success in many cases with just a few hundred milliseconds, but your mileage may vary. 

I hope this helps a little!

 

Take care,

Mike

Highlighted
Super Contributor.
Super Contributor.

Re: G-Suite / Google Apps Driver Crashing on Group Creation

Jump to solution

Thanks for your response, Mike.  We are grateful for the work you are doing.  Bill

Highlighted
Knowledge Partner
Knowledge Partner

Re: G-Suite / Google Apps Driver Crashing on Group Creation

Jump to solution
Brilliant info.
Thanks.
0 Likes
Highlighted
Honored Contributor.
Honored Contributor.

Re: G-Suite / Google Apps Driver Crashing on Group Creation

Jump to solution

Hi!

For #2, I have solved this by having a second policy on sub-ctp that verifies that the user location has been updated before allowing the driver to continue.

So in my first policy, I issue a move-command with direct, and I also set the new OU as an operation property (named new-placement).

On output I flag move events to be able to catch any move errors on ITP.

Output rule:

<rule>
	<description>Flag move event to store result in ITP</description>
	<conditions>
		<and>
			<if-class-name mode="nocase" op="equal">UserEntry</if-class-name>
		</and>
	</conditions>
	<actions>
		<do-if>
			<arg-conditions>
				<and>
					<if-operation mode="nocase" op="equal">move</if-operation>
				</and>
			</arg-conditions>
			<arg-actions>
				<do-set-local-variable name="is-move-event" scope="driver">
					<arg-string>
						<token-text xml:space="preserve">true</token-text>
					</arg-string>
				</do-set-local-variable>
			</arg-actions>
			<arg-actions>
				<do-set-local-variable name="is-move-event" scope="driver">
					<arg-string>
						<token-text xml:space="preserve">false</token-text>
					</arg-string>
				</do-set-local-variable>
			</arg-actions>
		</do-if>
	</actions>
</rule>

 

On input I have this rule:

<rule>
	<description>Store result of move event</description>
	<conditions>
		<and>
			<if-operation mode="nocase" op="equal">status</if-operation>
			<if-local-variable mode="nocase" name="is-move-event" op="equal">true</if-local-variable>
		</and>
	</conditions>
	<actions>
		<do-set-local-variable name="move-event-result" scope="driver">
			<arg-string>
				<token-xpath expression="@level"/>
			</arg-string>
		</do-set-local-variable>
	</actions>
</rule>

My second policy on sub-ctp looks like this:

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE policy PUBLIC "policy-builder-dtd" "C:\NetIQ\idm\apps\Designer\plugins\com.novell.idm.policybuilder_4.0.0.201906141638\DTD\dirxmlscript4.7.3.dtd"><policy xmlns:jThread="http://www.novell.com/nxsl/java/java.lang.Thread">
	<rule>
		<description>Verify that moves has been commited in Google before proceeding</description>
		<comment xml:space="preserve">We need to do this as moves takes some time to be commited in Google. If the move is not commited in Google and we send a modify (with any attribute) the move might be reverted as the driver shim updates orgUnitPath on the user also on modify operations.</comment>
		<conditions>
			<and>
				<if-class-name mode="nocase" op="equal">User</if-class-name>
				<if-operation mode="nocase" op="equal">modify</if-operation>
				<if-op-property mode="regex" name="new-placement" op="equal">.+</if-op-property>
				<if-local-variable mode="nocase" name="move-event-result" op="equal">success</if-local-variable>
			</and>
		</conditions>
		<actions>
			<do-set-local-variable name="user-placement-verified" scope="policy">
				<arg-string>
					<token-text xml:space="preserve">false</token-text>
				</arg-string>
			</do-set-local-variable>
			<do-while>
				<arg-conditions>
					<and>
						<if-local-variable mode="nocase" name="user-placement-verified" op="not-equal">true</if-local-variable>
					</and>
				</arg-conditions>
				<arg-actions>
					<do-set-local-variable name="user-nodeset" scope="policy">
						<arg-node-set>
							<token-query class-name="User" datastore="dest" scope="entry">
								<arg-association>
									<token-association/>
								</arg-association>
							</token-query>
						</arg-node-set>
					</do-set-local-variable>
					<do-set-local-variable name="user-DN" scope="policy">
						<arg-node-set>
							<token-xpath expression="$user-nodeset/@src-dn"/>
						</arg-node-set>
					</do-set-local-variable>
					<do-set-local-variable name="user-placement" scope="policy">
						<arg-string>
							<token-parse-dn dest-dn-format="slash" length="-2" src-dn-format="slash" start="0">
								<token-replace-all regex="/" replace-with="\\">
									<token-substring start="1">
										<token-local-variable name="user-DN"/>
									</token-substring>
								</token-replace-all>
							</token-parse-dn>
						</arg-string>
					</do-set-local-variable>
					<do-if>
						<arg-conditions>
							<and>
								<if-op-property mode="nocase" name="new-placement" op="equal">$user-placement$</if-op-property>
							</and>
						</arg-conditions>
						<arg-actions>
							<do-set-local-variable name="user-placement-verified" scope="policy">
								<arg-string>
									<token-text xml:space="preserve">true</token-text>
								</arg-string>
							</do-set-local-variable>
						</arg-actions>
						<arg-actions>
							<do-set-local-variable name="sleep" scope="policy">
								<arg-string>
									<token-xpath expression="jThread:sleep(3000)"/>
								</arg-string>
							</do-set-local-variable>
						</arg-actions>
					</do-if>
				</arg-actions>
			</do-while>
		</actions>
	</rule>
</policy>

This is probably not foolproof, but so far this has worked good.

Best regards

Marcus

 

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.