Highlighted
cesar.fernandez Absent Member.
Absent Member.
343 views

Multifolder Flex path filter

I am trying to process data from a cifs repository, but I need to access some specific directories only:

\\access_to_share\2016\08\02\specific_path\file-201608.log.gz

But  \\access_to_share\2016\08\02\ is full of different directories/files, where my user doesn't even have permissions to access. So I cannot use agents[x].processfoldersrecursively (It will have tons of access errors, and the repo is huge so it will take a long time to scan for new files). And I don't want to process the whole repo, I just need to process the new files.

I have tried to use "date based rotation" feature enabled by agents[x].wildcard = \'yyyy'\'MM'\'dd'\specific_path\file-'yyyyMM'.log, but I can not get it to work, It crash with a java.lang.null exception and a "is not correct for date based rotation"

Labels (2)
0 Likes
2 Replies
rhope Acclaimed Contributor.
Acclaimed Contributor.

Re: Multifolder Flex path filter

Looks like you have the quotes in the wrong place, they should go around the slashes and static info, not the date placeholders

0 Likes
cesar.fernandez Absent Member.
Absent Member.

Re: Multifolder Flex path filter

Right, so i changed it to:

agents[0].foldertable[0].folder=\\\\path_to\\repo\\

agents[0].foldertable[0].wildcard='\\'yy'\\'MM'\\specific_file\\'*'\\file-'yyMM*'.LOG'

Then, this is what I can see on the logs:

... In dir [null] verifying [FILEWITHDATA.CONF.201501] with regex [\\16\\08\\SPECIFIC_FILE\\.*\\FILE-1608.*\.LOG]

[2016-08-02 11:28:36,217][DEBUG][default.com.arcsight.agent.ee.b][WildcardFileFilter] Initializing wildcard filter [\16\08\specific_file\*\file-1608*.LOG]. Corresponding regex [\\16\\08\\SPECIFIC_FILE\\.*\\FILE-1608.*\.LOG]

[2016-08-02 11:28:36,217][FATAL][default.com.arcsight.agent.util.o][run]

java.lang.NullPointerException

    at com.arcsight.agent.ee.e.a(e.java:69)

    at com.arcsight.agent.util.o.f(o.java:781)

    at com.arcsight.agent.util.o.c(o.java:366)

    at com.arcsight.agent.util.o.c(o.java:385)

    at com.arcsight.agent.util.o.run(o.java:320)

    at java.lang.Thread.run(Thread.java:745)

If I change the "folder" to include the dynamic part -'\\'yy'\\'MM'\\specific_file\\'- the logs will show that the path "\\path_to\repo\\yy\MM\specific_file" doesn't exist, so its clear that the path is static, and I can not even use a '*'

And the wildcard regex is only matched with the actual file, not with the full path.

Any ideas? The problem is that every time that the connector tries to read a subfolder where the user does not have read access, the scan begins again, so the connector is not able to locate any files to process.

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.