log file flex connector
I have been trying to play and understand how can I configure a flex connector to meet my need but much luck and I hope you could help and guide me with this.
I have a bash script that retrieve from internet a list. After it runs, I have a log file which could vary in size (sometimes it's bigger, sometimes not). The mapping goes like this:
My problem is that I can't find a way to configure the flex connector so that it can detect when a new log is created and it have to manage it.
I can't use rotationscheme=none, usealternaterotationdetection=false, since because the file may grow or can shrink, the flex can only see when the log decrease in size.
I have to try setup a log rotation in my script:
But for what I understand the flex use the highest index value (log.3) for the actual log, and in this case I don't know how to configure the the agents.logfilename parameter (<path>/log ?) and the agents.rotationschemeparams (log.'%01d,001,003,true' ?).
This thing is starting to male me crazy!
Any help will be very appreciated.
Re: log file flex connector
I am going to reply to myself. I think I was able to solve my issue (still I need to do additional testings ). Instead of using the Log File FlexConnector, I am currently plays with the Folder Follower Flex Connector (option 17 during runagentsetup in the flex connector setup).
I run the folder follower in batch mode. Other settings that I have changed are:
(The script generate a file called list.csv.x where x is a number starting with 1)
So my script create at the end a file called list.csv.1
The connector scan the folder, found the file because it matches the pattern and then process it and delete the file at the end.
If I run the run another time, even if the file size has changed (inc or decrease), the file is processed anyway.
I'm almost happy . I will try to play with file rotation a little.