test4echo1 Absent Member.
Absent Member.
293 views

Pre-persistence rule field population fails

I wrote a pre-persistence rule in ESM 6.5sp1.

Our URL filtering product populates the Request URL field.

The CEF data (read from a CEF File) is in the format

          request="static.ess.apple.com/connectivity.txt"

In an active channel, I see the double quotes around the requestUrl field contents.

          "static.ess.apple.com/connectivity.txt"

I was hoping to achieve two things with the rule.

     1) extract the requestUrlAuthority from the requestUrl (most important).

     2) get rid of the double quotes in the URL (if possible).

I used local variables to populate the requestUrlAuthority.

     firstSlash = IndexOf(requestUrl,"/")

     reqUrlAuth = Substr(requestUrl,1,firstSlash)

Then I set the 'Every Event > Set Event Field Actions'

     requestUrlAuhority = $reqUrlAuth

The requestUrlAuthority field populates correctly, but the original requestUrl now shows //static.ess.apple.com

Worse yet - everything after the '.com/' is gone and the double quotes at the beginning are now // marks and none at the end.

Since I did not put a value requestUrl field - I am not sure why it changes on every record.

I attempted setting a local variable to concat the requestUrl with (single or double) quotes - before using the new local variable in the index & substr variables above ... but it makes things worse in both fields.

I have not event attempted to remove double quotes from the requestUrl ....

Our URL product does not provide the CEF 'request' fields (authority, file name, port, etc.).

Some content we wish to develop really needs these fields.

Any ideas?

0 Likes
Reply
5 Replies
rhope Acclaimed Contributor.
Acclaimed Contributor.

Re: Pre-persistence rule field population fails

I think the problem is that once you set requestUrlAuthority ESM is trying to build requestUrl from requestProtocol, requestUrlAuthority etc hence the // at the beginning

From the flex developers guide:

The connector returns a URL when requestUrl is invoked. The URL is stored in the event

table. ESM can then parse the URL to derive the following URIs

  • requestProtocol
  • requestUrlAuthority
  • requestUrlHost
  • requestUrlPort
  • requestUrlFileName
  • requestUrlQuery

requestUrl has the following format:

<protocol>://<authority>@<host>:<port>/<filename>?<query>

Do not set a value for requestUrl and set a value for one or more of the URIs.

I know the above is for flex connectors and you're working in ESM, but from (a vague) memory I think the same applies. If I understand correctly either the requestUrl can be built from the components OR the components can be extracted from requestUrl. So if the requestUrl is set correctly the other fields should be populated where available.  I'd try just removing the quotes and see if the other fields are extracted from the requestUrl field

0 Likes
Reply
Highlighted
test4echo1 Absent Member.
Absent Member.

Re: Pre-persistence rule field population fails

I removed the double quotes as suggested.

Several request fields are populating automatically.

So far, I see request... (Context, UrlFileName, & URLQuery).

I still do not see UrlAuthority.

Could it be due to missing http:// or https:// ?

0 Likes
Reply
rhope Acclaimed Contributor.
Acclaimed Contributor.

Re: Pre-persistence rule field population fails

Looking at your original url I think you're actually looking for requestUrlHost i.e. static.ess.apple.com is the hostname, authority would capture things like ftp username

0 Likes
Reply
test4echo1 Absent Member.
Absent Member.

Re: Pre-persistence rule field population fails

That is probably more appropriate, however I am not getting values for either requestUrlHost or requestUrlAuthority.

0 Likes
Reply
rhope Acclaimed Contributor.
Acclaimed Contributor.

Re: Pre-persistence rule field population fails

Testing it now, it looks like the protocol is required for the extraction to work correctly.  It also looks like both the requestUrlAuthority and requestUrlHost will be populated with static.ess.apple.com using your example above. If you go the other way and populate the requestUrlHost and requestUrlFileName you'll get //static.ess.apple.com/connectivity.txt.

0 Likes
Reply
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.