flexconnector for file, file loaded every 5 min, is there any chance to read only the new entries?
I am going to develope a flexconnector to read a file which comes from an output of a sql script. This script runs every 5 minutes and the old file delets and a new file (same name and with the concatenate new entries) in the directory reloaded. My question is, how i can parse just the new entries not reading the file from the beginning which had beed read in the old file? is ther any way to workaround this so a flexconnector parse the file and follow the new lines in the reloaded file?
Thank you in advance for your respose
Thank you Sumanth,
Here the file is not the same file, the file has been replaced when the new but with the same name is loaded. i think Connector put a flag at the end of file when it's been read and if the new line added to it then the connector realize it and process. But in this case the new file has no flog on it, I suppose.
The property you need is the preservestate. This defaults to false, but you must set it to true so that the smartconnector remebers where it was in processing the file.
Below is copied from Flex Connectors Developers Guide Appendix G:
If set to true, remembers the last location read in the file periodically,
depending on the values set for the perservedstatecount and
If set to false, then nothing is written and the connector has no record of
where it left off. In this case, the values of perservedstatecount and
preservedstateinterval are ignored.
I would add that I have done this in exactly the same circumstances where an SQL script overwrote the file (in my case every hour) and it worked perfectly. BUT have you thought of how you will roll over the file every day? The connector can throw an error if a file which was say 800 records long at 23:59, suddenly becomes 2 records long at 00:04.
Thank you for your responses.
I implemented it as a multifilereader connector. The scripts runs every 15 miutes and a new file with unique details like yyyyhhss (file_xxxxxxxx) will be created in the dirctory with only the last 15minutes data.
Tested and No problem then with the dublicate or rotating 🙂
all the bests//Fred