424 views

Flex connector reading duplicate logs ?

Hello guys,

I created a flex connector for reading log file from multiple folders. Each time when i load the log file it read whole file however i did not updated the file content but the connector read it  form the beginning each time i load it to the connector.  😞  

Below is the agent.properties parameters : 

agents[1].destination.count=1
agents[1].destination[0].agentid=3aE0bUG8BABCAQg3putMKfw\=\=
agents[1].destination[0].failover.count=0
agents[1].destination[0].params=<?xml version\="1.0" encoding\="UTF-8"?>\n<ParameterValues>\n <Parameter Name\="cefver" Value\="0.1"/>\n <Parameter Name\="port" Value\="443"/>\n <Parameter Name\="host" Value\="Mylogger.company.com"/>\n <Parameter Name\="rcvrname" Value\="T_DB"/>\n <Parameter Name\="compression" Value\="Disabled"/>\n</ParameterValues>\n
agents[1].destination[0].type=loggersecure
agents[1].deviceconnectionalertinterval=60000
agents[1].enabled=true
agents[1].entityid=JlAbUG8BABCAQw3mutMKfw\=\=
agents[1].fcp.version=0
agents[1].foldertable.count=1
agents[1].foldertable[0].badsubfolder=bad
agents[1].foldertable[0].configfile=timesten
agents[1].foldertable[0].configfolder=/opt/arcsight/connectors/connector_3/current/user/agent//flexagent/
agents[1].foldertable[0].configtype=sdkrfilereader
agents[1].foldertable[0].delay=10000
agents[1].foldertable[0].encoding=
agents[1].foldertable[0].extractfieldnames=
agents[1].foldertable[0].extractregex=
agents[1].foldertable[0].extractsource=File Name
agents[1].foldertable[0].fixedlinelength=-1
agents[1].foldertable[0].fixedlinelengthcontains=Fixed Number Of Characters
agents[1].foldertable[0].folder=/tmp/logs/server1
agents[1].foldertable[0].followexternalrotation=false
agents[1].foldertable[0].maxretries=-1
agents[1].foldertable[0].minfilelenght=-1
agents[1].foldertable[0].mode=DeleteFile
agents[1].foldertable[0].modeoptions=processed
agents[1].foldertable[0].monitoringinterval=30000
agents[1].foldertable[0].onrotation=None
agents[1].foldertable[0].onrotationoptions=processed
agents[1].foldertable[0].preservestate=true
agents[1].foldertable[0].processfoldersrecursively=false
agents[1].foldertable[0].processinglimit=256
agents[1].foldertable[0].processingmode=realtime
agents[1].foldertable[0].processingthreshold=3600000
agents[1].foldertable[0].processingtimeout=120000
agents[1].foldertable[0].retryinterval=1000
agents[1].foldertable[0].sleeptime=5000
agents[1].foldertable[0].startatend=false
agents[1].foldertable[0].triggerextension=.done
agents[1].foldertable[0].usealternaterotationdetection=true
agents[1].foldertable[0].usefieldextractor=false
agents[1].foldertable[0].usenonlockingwindowsfilereader=false
agents[1].foldertable[0].usetriggerfile=false
agents[1].foldertable[0].wildcard=*
agents[1].id=3aE0bUG8BABCAQg3numMKfw\=\=
agents[1].internalevent.filecount.duration=-1
agents[1].internalevent.filecount.enable=false
agents[1].internalevent.filecount.minfilecount=-1
agents[1].internalevent.filecount.timer.delay=60
agents[1].internalevent.fileend.enable=true
agents[1].internalevent.filestart.enable=true
agents[1].persistenceinterval=0
agents[1].preservedstatecount=10
agents[1].preservedstateinterval=30000
agents[1].type=sdkmultifolderreader
agents[1].unparsedevents.log.enabled=true

 

Regards,

Miran. 

0 Likes
3 Replies
Commodore
Commodore

Hi @Miran Arsalan Sleman 

Try changing the below parameter and restart the connector service

agents[1].foldertable[0].startatend=true

Hope this helps!

 

Regards

Ajith K S

0 Likes
Lieutenant
Lieutenant

I also have:
agents[0].preservestate=true
agents[0].preservedstatecount=5
agents[0].preservedstateinterval=30000
added it when I was out of ideas and mine works
Captain Captain
Captain

Change the parameters to below and try :

  • startatend=true
  • preservestate=false
  • processingmode =realtime
  • followexternalrotation=true
  • rotationscheme=None
  • usealternaterotationdetection=true

Regards,

Sumanth

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.