renjithrjames Absent Member.
Absent Member.
303 views

Folder Regex Reader Current Event

Good Day!

In our environment , We have a requirement to integrate a application which is recording the data to flat file. In this same file it already having 3 year data but we don't want read older data (complete logs in the file), which the requirements say activities only from the current date.Is there any possibility to say the folder agent to take log starting from current day.

Appreciate quick help!

Thanks

Renjith R James

Labels (4)
0 Likes
Reply
5 Replies
renjithrjames Absent Member.
Absent Member.

Re: Folder Regex Reader Current Event

Expecting Support...

0 Likes
Reply
Micro Focus Expert
Micro Focus Expert

Re: Folder Regex Reader Current Event

That depends on a few things.

- how often is the file updated? If there is one file that has events from more than 3 years, this must be a very low event rate. Most solutions have a file rotation that will rotate the file out after it was read by the connector, maybe using date in the file name. Then the connector reads only the current file.

If you are only doing this once, can you manually change the files so the current file has the current events?

There are other things you could use too. For example the regex file reader lets you specify

line.ignore.regex=put here the regex that matches dates before the current date

For example if your dates are 02-Jan-2014 and 03-Mar-2015 then you could do

line.ignore.regex=(\d{1,2}-\S{3}-2014 | \d{1,2}-Jan-2015| and so on)

0 Likes
Reply
renjithrjames Absent Member.
Absent Member.

Re: Folder Regex Reader Current Event

Hi Aaron,

Thanks for your sharing. I am afraid if this parameter utilize the process highly hence it is one of the mission critical system. However, as you mentioned, we are on the discussion to rotate file on daily basis.

Thanks & Regards

Renjith R James

0 Likes
Reply
Highlighted
Outstanding Contributor.. Pushpendra_Rathi Outstanding Contributor..
Outstanding Contributor..

Re: Folder Regex Reader Current Event

Hi Renjith

Try the following:

1) You can use/configure the "agents[x].startatend = true" parameter in agent.properties file...

agents[x].startatend

The default is true. Useful when log files to be processed already exist and contain data at connector startup or when the log file rotation takes place. Setting this value to false will cause the entire file to be read at every startup, which could lead to duplicate events. So you can set this value to true so that the connector will start reading the file from the END (after getting newer events) and not from beginning...

2) And you can also use the parameter "agents[x].preservestate=true"

agents[x].preservestate=true

it marks the file in the location last read at shutdown and begins to read the file from that location at restart means This will tell the SmartConnector to remember the last record that was processed and then when new records are added it will grab them.

Regards

Pushpendra

0 Likes
Reply
renjithrjames Absent Member.
Absent Member.

Re: Folder Regex Reader Current Event

Hi Pushpendra,

Thanks for your help, in addition to above parameters. The same folder is having archiving events also, every hour the log file is archieving in .gz extension and so I modified the wildcard entry as *.log (because the current log file is .log format), and after the parameter change it is not picking the log. Is there any parameters to track file modification ?(I tried parameter "agents[x].foldertable[x].followexternalrotation=true" which doesn't work).

Thanks & Regards

Renjith

0 Likes
Reply
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.