ALERT! The community will be read-only starting on April 19, 8am Pacific as the migration begins. Read more for important details.
ALERT! The community will be read-only starting on April 19, 8am Pacific as the migration begins.Read more for important details.
3581 views

Populating an Active List without Rule?

Hi,

Is it be possible to populate an AL without using  a Rule ? I would like to import data from a .csv file directly into an AL regularly by script. Is there a way to do that? I know of the possibility to import via the ESM Console but i want to do it from a regularly scheduled shell script.

regards,

Steven

Labels (1)
Tags (1)
0 Likes
26 Replies
Absent Member.
Absent Member.

I believe the model import connector in identityview works this way.

0 Likes
Absent Member.
Absent Member.

Several people at the last user conference were talking about being able to do what you are talking about with some xml based stuff. Believe it might have come up in one of the presentations but can’t remember which one.

0 Likes

Found one way to do it:

Create an XML file and use the archive tool to import the AL

On another threat i found how to insert/delete entries by using <activeListEntries> to overwrite, <insertListEntries> to add and <deleteListEntries>to delete

https://protect724.arcsight.com/message/15143#15143

Would be better though if we could dump a CSV into an AL directly from the shell.

thnx

steven

0 Likes
Cadet 2nd Class Cadet 2nd Class
Cadet 2nd Class

Check out Raju's ESM Tips & Tricks presentation from Protect '10.  He talks about how to do it without using rules.

The preso is here:https://protect724.arcsight.com/docs/DOC-1405

Unfortunately, it's only available if you went to the conference.

0 Likes
Cadet 2nd Class Cadet 2nd Class
Cadet 2nd Class

Hey Steven,

yes. As Damian said, our MIC works that way.

Basically you create a connector (in your case a CSV reader) and integrate a velocity script that generates XML files which are then pushed to the web services API of the manager.

I've done this once for a trial to import ID data from Ldap exports from eDirectory.

The challenge is that it is fully undocumented.

Have you been at the last User Conference? I beleive Raju had a presentation on this which might be the best way to get started.

Best regards,

Till

0 Likes

tnx all,

Found the doc, its on pg. 26 of Raju's presentation and further. We'll try that, we're now using the archive tool to import a generated XML file, but as stated in the doc it can be problematic with large amounts of data.

--off-topic

Would be very usefull though if we had a more programmable interface with the Smart Connector framework. An API of some sort with full scriptable abilities. This would enable us to talk to the connector in stead of having to use external java, python or perl for search, extraction, conversion, reformatting and several time / mathematical taks with logfiles/feeds.

Feature request to integrate a full OO script language in the flex connector ???

Shouldn't be so hard to have a standard java/python class lib available in the flex connectors? Java is already in there....

grtz,

steven

0 Likes
Absent Member.
Absent Member.

Has anyone used Raju's method of using a flexconnector with a velocity macro successfully?

I am following the steps outlined in the presentation, but am unable to get it to work. I am using the latest smartconnector software (ArcSight-5.2.2.6221.0-Connector-Win.exe) where I think the agent.component[34].maxeventsbeforebuild=20000 and agent.component[34].buildmodeldelay=90000 appear to be updated to [35] now, but I'm not sure. I've tried it both ways, but still no luck.

I don't seem to get any errors from the flexconnector, so it appears to be parsing the file correctly; but I do get errors in my agent.log file "java.lang.NullPointerException" so maybe the velocity macro isn't working correctly??

I also don't see any files in the $managerDir\archive\webservice directory like Raju points out to look for.

Are there pre-requisites to getting this method to work?

Any help would be appreciated.

0 Likes
Absent Member.
Absent Member.

I was finally able to get this working. Hopefully these notes help some other person trying this method:

I am not exactly sure what I did to get it to work, but I think that I had a problem with originally thinking I could have anything as my event.deviceProduct and event.deviceVendor mapping.

I ended up copying Raju's mappings:

event.deviceVendor=__stringConstant(ArcSight)

event.deviceProduct=__stringConstant(FlexArchiveImport)

I also added the following lines to my agent.properties file:

agents[0].component[35].maxeventsbeforebuild=200
agents[0].component[35].buildmodeldelay=9000

and also added DEBUG mode to my smart connector in the same agent.properties file to help me troubleshoot:

log.global.debug=true
log.channel.file.property.package.com.arcsight=0

In my velocity macro (ips.vm) I ended up using the shortened XML example without the XML header and closing tag.

I did notice that xml files finally showed up in my $managerDir\archive\webservice folder on my ESM. (not sure the significance of these files or how they are created, but they finally came through.)

FYI there is a "final" version of Raju's presentation that had better information found here: https://protect724.arcsight.com/docs/DOC-1835

Best of luck to anyone else trying this method

0 Likes
Vice Admiral
Vice Admiral

Hi everyone!

I've done some testing with this today. I cannot get the *.vm to be picked up by the newer SmartConnectors. I've test 5.2.1 and 5.2.3 and neither worked! NullPointerException continously, which looks like the velocity macro isnt being picked up.

Tried it with 5.1.7 without any of the properties file tweaks mentioned and it worked first time.

Will investigate further...

Tom

0 Likes
Absent Member.
Absent Member.

Yes, sorry I forgot to mention. We noticed the exact same thing. 5.1.7 smartconnector was the only one that would work. Not sure if the functionality was removed or if there is a bug.

Maybe a good talking point for this year's protect conference?

0 Likes
Cadet 2nd Class Cadet 2nd Class
Cadet 2nd Class

Care to comment Raju? I've been trying to get this to work with 5.2.3 and no luck as the posters above describe. No errors in the local agent logs for me, the input csv gets successfully processed, but nothing shows up in the webservice folder on the manager

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.