Populating an Active List without Rule?
Is it be possible to populate an AL without using a Rule ? I would like to import data from a .csv file directly into an AL regularly by script. Is there a way to do that? I know of the possibility to import via the ESM Console but i want to do it from a regularly scheduled shell script.
Several people at the last user conference were talking about being able to do what you are talking about with some xml based stuff. Believe it might have come up in one of the presentations but can’t remember which one.
Found one way to do it:
Create an XML file and use the archive tool to import the AL
On another threat i found how to insert/delete entries by using <activeListEntries> to overwrite, <insertListEntries> to add and <deleteListEntries>to delete
Would be better though if we could dump a CSV into an AL directly from the shell.
Check out Raju's ESM Tips & Tricks presentation from Protect '10. He talks about how to do it without using rules.
The preso is here:https://protect724.arcsight.com/docs/DOC-1405
Unfortunately, it's only available if you went to the conference.
yes. As Damian said, our MIC works that way.
Basically you create a connector (in your case a CSV reader) and integrate a velocity script that generates XML files which are then pushed to the web services API of the manager.
I've done this once for a trial to import ID data from Ldap exports from eDirectory.
The challenge is that it is fully undocumented.
Have you been at the last User Conference? I beleive Raju had a presentation on this which might be the best way to get started.
Found the doc, its on pg. 26 of Raju's presentation and further. We'll try that, we're now using the archive tool to import a generated XML file, but as stated in the doc it can be problematic with large amounts of data.
Would be very usefull though if we had a more programmable interface with the Smart Connector framework. An API of some sort with full scriptable abilities. This would enable us to talk to the connector in stead of having to use external java, python or perl for search, extraction, conversion, reformatting and several time / mathematical taks with logfiles/feeds.
Feature request to integrate a full OO script language in the flex connector ???
Shouldn't be so hard to have a standard java/python class lib available in the flex connectors? Java is already in there....
Has anyone used Raju's method of using a flexconnector with a velocity macro successfully?
I am following the steps outlined in the presentation, but am unable to get it to work. I am using the latest smartconnector software (ArcSight-184.108.40.20621.0-Connector-Win.exe) where I think the agent.component.maxeventsbeforebuild=20000 and agent.component.buildmodeldelay=90000 appear to be updated to  now, but I'm not sure. I've tried it both ways, but still no luck.
I don't seem to get any errors from the flexconnector, so it appears to be parsing the file correctly; but I do get errors in my agent.log file "java.lang.NullPointerException" so maybe the velocity macro isn't working correctly??
I also don't see any files in the $managerDir\archive\webservice directory like Raju points out to look for.
Are there pre-requisites to getting this method to work?
Any help would be appreciated.
I was finally able to get this working. Hopefully these notes help some other person trying this method:
I am not exactly sure what I did to get it to work, but I think that I had a problem with originally thinking I could have anything as my event.deviceProduct and event.deviceVendor mapping.
I ended up copying Raju's mappings:
I also added the following lines to my agent.properties file:
and also added DEBUG mode to my smart connector in the same agent.properties file to help me troubleshoot:
In my velocity macro (ips.vm) I ended up using the shortened XML example without the XML header and closing tag.
I did notice that xml files finally showed up in my $managerDir\archive\webservice folder on my ESM. (not sure the significance of these files or how they are created, but they finally came through.)
FYI there is a "final" version of Raju's presentation that had better information found here: https://protect724.arcsight.com/docs/DOC-1835
Best of luck to anyone else trying this method
I've done some testing with this today. I cannot get the *.vm to be picked up by the newer SmartConnectors. I've test 5.2.1 and 5.2.3 and neither worked! NullPointerException continously, which looks like the velocity macro isnt being picked up.
Tried it with 5.1.7 without any of the properties file tweaks mentioned and it worked first time.
Will investigate further...
Yes, sorry I forgot to mention. We noticed the exact same thing. 5.1.7 smartconnector was the only one that would work. Not sure if the functionality was removed or if there is a bug.
Maybe a good talking point for this year's protect conference?