Highlighted
Super Contributor.. dbarry1 Super Contributor..
Super Contributor..
424 views

Flex connector problems parsing MM/DD format near year-end

(Note: I edited the title and messagine part to clarify that the incoming fomat is MM/DD (without a year), which is common in syslog files).

This question was originally directed to support, but  was punted as "beyond the basic framework."    I'll post my original exact question, with some additional information.

I am trying to write a flex connector that will read log files that contain dates that are only specified in MM/DD HH:MM format. (Theyear is not specified).

We have determined using a definition like this will convert this date to a date:

token[1].name=Date

token[1].type=TimeStamp

token[1].format=MM/dd HH\:mm

However, we are concerned that processing at year end will be incorrect due (for example, a log containing a date of 12/30 processed on january 1 will have have the year of processing, not the proper year.

 

(NOTE:   I later demonstrated this was more than a suspicion, it was a fact.)

On page 101 of the flex developer guide, there is a possible clue on how to address these files, the __setYeartoCurrentYear function. It says:

<QUOTE>__setYeartoCurrentYear TimeStamp

The parameter is a single TimeStamp, for which the year is forcibly set to the current year, plus or minus one (depending in part on the

syslog.future.limit property). This is usedfor TimeStamps that do not have a defined year. </QUOTE>

 

This sounds like what we probabbly want to use. I imagine the syslog.future.limit property is some definition of to interpret the year (i.e, within the first nn days of the year, consider the late months to belong to the prior year.)

 

However, I can find no documentation of the syslog.future.limit proprty anywhere else. Assuming my guess is correct, I need to know where this property is used, and what are the parameters. I have searched the documentaion and the KB with no results.

Can we get a pointer to the documentation of this property and/or a clear explanation of how we can convert a date provided in mm/yy format to a full date that allows us to specify a year rollover boundary.

======

After posting this, support said it should not be a problem, and ignored my question about the syslog.future.limit property.

I then sent them test files showing that it most certainly was a problem, and can I please get an answer to my original question.

Here was the last reply.

Thank you for the update.

Unfortunately, I provided the best possible test scenario possible and it did not seem to provide the results I had hoped for.

Addtionally, the ArcSight Support team is not able to support the flex connector's issue beyond the basic connector framework.

Thus, the questions that you have asked are beyond our scope.

However, there are resources that you may take:

1. ArcSight Profession Services can be commissioned to assist you with your flex connector deployments

2. ArcSight forums may provide you with answers to you questions

I regret no able to assist you further. However, if you have any further questions, please let know.

-===================

Ok, so support has punted to you, noble forum. 😉   Any takers.   I'm sure my original message contains the clue I need, but again, the function is not documented in a way that explains how it may be used.   Im sure I cdould spend a day experimenting, but if someone has already unravelled the secret,  I would appreciate a clue.... 😉

 

Labels (3)
0 Likes
Reply
2 Replies
deathbywedgie1 Frequent Contributor.
Frequent Contributor.

Re: Flex connector date parsing MM/YY problems near year-end

I was confused at first... your subject line and part of your description says MM/YY when I think you mean MM/dd.

In all seriousness, I've got a flex connector on the back burner because I haven't had time to figure out how to tackle this. It should be easy for either a property to be created to control how this behaves or to simply edit the behavior of the flex connector framework... when an event from the month of December is received in January, it should simply assume that the event is from the previous year. I recommend submitting this as a feature request instead of a support case. At a minimum there should be a connector setting to enable this kind of logic even if they decide not to make it mandatory logic on the connector's part.

On a side note, I suspect there MIGHT be a way to play around with operations in the flex connector that will do some logical work to determine whether the month in the event is greater than the current month by 'x' amount, allowing you to map to the previous year instead of the current year. I'm not sure how you'd go about doing that, though.

If you figure this out, please share!

0 Likes
Reply
Super Contributor.. dbarry1 Super Contributor..
Super Contributor..

Re: Flex connector date parsing MM/YY problems near year-end

Yeah, there are several "hints" that such functionality exists --  see __parseMutableTimeStamp and __setYeartoCurrentYear.   But the needed info is missing -- how (and where) to use syslog.future.limit.

0 Likes
Reply
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.