Sentinel Collector SDK 2014 Updates: Development Tests

Sentinel Collector SDK 2014 Updates: Development Tests

For the past while this series has been covering some of the powerful updates present in the 2014 Preview version of the new Sentinel SDK, which is used for creating various types of Sentinel plugins (collectors, reports, solution packs, actions, etc.) and so far we have been focusing on the collector portion. Today is no different, as we'll be covering development tests for our new, shiny collector plugins.

Hopefully the purpose of test cases is pretty obvious, but a lot of the time the work to create them does not seem to be justified by what you get out from them. Creating test cases is time-consuming, and it may feel like all you get is a nice 'OK' or a green line on a huge table of output indicating something didn't break. Big deal, you may think... it never does break. Of course, the benefit of tests around any software product becomes much more evident as that product grows, and as complexity increases. Testing individual components in an automated way without any work on your part means that you can focus on the real problem, or that you can find a regression in code immediately without needing to do a lot more manual work to get to the same destination as the test's target.

Working in complex IT environments, and in particular writing scripts and doing other logic-creation in the Identity Management field, it's pretty clear how quickly a small typo can lead to hours of one's life wasted tracking down something silly; if only there had been a simple test case based on known input, known output, automatically running when changes happened, so much time could have been saved. Perhaps more-importantly, when that last-minute change came from the customer for something seemingly-insignificant, so much more confidence could have been placed in the final result that the change was truly insignificant to proper function of the code, or that other problems could be found immediately. Confidence in code is important, and last-minute requests from customers are almost the norm; having test cases lets you handle the latter while keeping the former, along with your sanity.

From here I'm going to assume that you already have a collector setup in Eclipse using the 2014 Preview SDK as discussed previously; if not, check out previous articles and get that going. A complex collector is not needed for any of this, and in fact a basic skeleton automatically created by the SDK will suffice for starters and is what I will be using just so we can all start from the same code. Once setup, and the new default collector template is used to create your own new collector, we can immediately start running test cases. Yes, running, not creating, because the friendly folks at NetIQ have shipped test cases for things that should apply to all collectors everywhere. What could possibly apply to all collectors? A few examples may help, so I'll focus on Sentinel and XDAS Taxonomy first, as it's a good example.

As a collector-setup note, do the following with your new skeleton collector if not done already. Within Eclipse open the connectionMethods.xml file, which is where you define which connection methods will be supported by this collector. Uncheck all of the options except for Syslog and File (be careful as there is a missing scrollbar so if you do not see SDEE at the bottom, make the window bigger, or hack the XML). Also, under File uncheck the 'Enable' checkbox next to 'Line'. Finally, leave ONLY 'Application ID' enabled under the Syslog Connector and set the AppID as 'ABApplication' (or some other value that you'll replace 'ABApplication' with throughout this article in your environment). Save and close the tab and your collector should be ready for those connection method. We will come back to this file shortly, but first we need to talk more about tests.

In the Sentinel world the XDAS standard is followed for classification of events into world-agree-upon buckets. A Sentinel website outlining the XDAS options for its evens can be found at https://www.novell.com/developer/plugin-sdk/sentinel_taxonomy.html and basically lets one easily see, based on a given event or outcome, the various XDAS fields' settings. If you have an event which gets the XDASTaxonomyName field populated with 'XDAS_AE_CREATE_ACCOUNT" then that means eight other fields are set in ways that are predictable and useful within Sentinel (and other downstream systems potentially) for searching, reporting, correlation, etc.

XDASRegistry = 0
XDASProvider = 0
XDASClass = 0
XDASIdentifier = 0
TaxonomyLevel1 = SYSTEM
TaxonomyLevel2 = USER
TaxonomyLevel3 = CREATE
TaxonomyLevel4 = [blank]


Having all of these options coded into a collector for every type of event (meaning setting all of these options for every type of event, plus the options for the XDAS Outcome of the event) is painful, which is why we have the XDASTaxononyName option which will map to the eight other values. All one needs to do, then, is make sure that all events of a certain type map correctly to the appropriate XDASTaxononyName, which is done with taxonomy mapping within the collector. The taxonomy.map file, then, is consulted during the normalize() function of the collector to apply the correct taxonomy based on the key sent to the taxonomy.map file, which may look like this:

~TaxonomyKey,XDASTaxonomyName,XDASOutcomeName
Add,XDAS_AE_CREATE_ACCOUNT,XDAS_OUT_SUCCESS
Modify,XDAS_AE_MODIFY_ACCOUNT,XDAS_OUT_SUCCESS
Add_Attribute,XDAS_AE_MODIFY_ACCOUNT,XDAS_OUT_SUCCESS
Modify_Attribute,XDAS_AE_MODIFY_ACCOUNT,XDAS_OUT_SUCCESS
Start,XDAS_AE_ENABLE_SERVICE,XDAS_OUT_SUCCESS
Stop,XDAS_AE_TERMINATE_SERVICE,XDAS_OUT_SERVICE_UNAVAILABLE
Error,XDAS_AE_DISABLE_SERVICE,XDAS_OUT_SERVICE_ERROR


For the developer, all that needs to happen is for the key to be sent once per event with a value of 'Add', 'Modify', 'Add_Attribute', or 'Modify_Attribute', and then Sentinel applies the XDASTaxonomyKey as well as XDASOutcomeName, then filling in about ten other fields automatically. So much work is saved by having this in place, but what if something in the SDK incorrectly sets those ten fields? What if I change my taxonomy key at some point for a particular event (oh, that was an enable, not a create; fixed) but forget to update other values downstream? What if I add a last-minute change and completely forget taxonomy so that the classification just does not happen? All of these things can be detected out of the box and it is these types of things that the SDK's default tests will handle, meaning the work that I do, and that you do, and that NetIQ developers all do the same way, is all done for us (by the NetIQ developers).

Another example of boilerplate tests is around plugins having appropriate fields populated with good default values, such as the TenantName which should be set to 'default' normally. In older collectors, this was set to 'unknown' by default, so putting an older collector into the new SDK will immediately help you by seeing this change in what is correct out of the box. Other fields that should be set all of the time include 'ObserverIP' and 'ObserverHostName'; if neither is set, that's an error, and the default tests tell you as much. Verifying the syntax of collector input files (taxonony.map, xdas_out.map and xdas_tax.map) is also done so that if you do something silly like leave trailing spaces at the ends of lines in map files you'll be warned from the start. If you forget a Description for your collector, or if you try to have multiple default connection methods (there can be only one default of course), or if you somehow try to include a file in the collector but forget to put that file into the collector itself..... all captured by default tests. Neat, right? This is all probably the kind of thing that was added as it was helping catch little niggling problems for the NetIQ devs, so it is logical to assume they could hit us, and therefore tests in the SDK similarly benefit us.

Before I forget, the output from tests is shown in the Console at the very end of the testing process. Text will scroll for a while, probably close to a minute by default including build and test time, and then a line like this will show up:

[move] Attempting to rename: /home/ab/code/sentinel-plugin-sdk/content/build/Collector/ABSoftware_ABApplication_2011.1r1-201501081033-internal-test/tmp/Test-Results_ABSoftware_ABApplication_2011.1r1-201501081033-internal-test.html to /home/ab/code/sentinel-plugin-sdk/content/build/Collector/ABSoftware_ABApplication_2011.1r1-201501081033-internal-test/Test-Results_ABSoftware_ABApplication_2011.1r1-201501081033-internal-test.html


The latter path is the test output, so just copy/paste in a browser to open it and then you can see the results I'm referring to throughout this note. Did you know that you can customize where build directories are created rather than leaving them in the default location? Look in ~/.netiq/pluginsdk/dev.properties to set the buildroot to something other than the default, perhaps to a shorter path for example, or to something on your user's desktop.

One of the best results of just running the basic tests, even before trying to use the debugger to step through code, is that you'll end up with dozens of nasty errors if you do not provide input data, which is easy to forget that you need your first time using the new SDK since the previous way of using the debugger allowed sending events to Sentinel from outside a lot of the time. Those errors look like this:

An Exception occurred. The Test has been terminated.
Java Exception JavaException: java.io.FileNotFoundException: /home/ab/code/sentinel-plugin-sdk/content/build/Collector/ABSoftware_ABApplication_2011.1r1-201501080639-internal-test/tmp/data.in (No such file or directory) Test Result: error


This may not be as clear as "You didn't provide any input data in your variety.dump file, silly" but the lack of a file named 'data.in' hopefully is clear enough, especially since there are dozens of these errors in a row. There are some successful tests, around the plugin itself, but anything requiring actual input data failed because there were no input data. If you were in the new built-in debugger (see a previous article in this series) and had just set breakpoints and clicked 'Play' but had never seen an event, well, you would have wasted a bit of time waiting for those events. Here, at least, there are some pointers. I would not recommend running tests before every use of the Debugger, but having another way to quickly verify things are sane is really nice. I hope this may do a nice job of catching fatal syntax problems that are not otherwise caught by the build process, though I have not tested that yet.

One more thing may stand out at the top of this test report assuming you are using the same build of the SDK as me, and that is this 'Plug-In Tests' results, again from NetIQ:

13599891 - Verify whether Stateful property is defined for every ConnectionMode other than File Replay mode - 0 events passed out of 1 events matching the Match Condition: Failed test on : "The Stateful property must be defined. It" was "0" but was expected to be "12". Test Result: fail


This error was one that I saw even with my final collector until I talked about it with NetIQ. It turns out that this is an SDK bug, and not in the test. The default SDK should be setting the Stateful property on syslog connection methods, but it is not, and thankfully there are test cases in the SDK to easily identify this omission. Bug# 909105 was entered against the SDK and the following XML can be added to the appropriate connection methods to bypass the error:

<Property>
<Name>Stateful</Name>
<Value>false</Value>
</Property>


To implement the fix we'll now go back to the connectionMethods.xml file in Eclipse as I mentioned we would above:

<ConnectionMethod>
<ConnectorName>SYSLOG</ConnectorName>
<IsDefault>1</IsDefault>
<!-- select one of the following modes and remove the others -->
<ConnectionModes>
<ConnectionMode>
<InternalName>map</InternalName>
<DisplayName>Syslog:Map Output (appid)</DisplayName>
<Description>This mode outputs syslog data in "map" mode, meaning that header information is presented in special map variables.</Description>
<IsDefault>1</IsDefault>
<Properties>
<Property>
<Name>DataFormat</Name>
<Value>map</Value>
</Property>
<Property>
<Name>Applications</Name>
<Value>ABApplication</Value>
</Property>
</Properties>
</ConnectionMode>
</ConnectionModes>
</ConnectionMethod>

to this:
<ConnectionMethod>
<ConnectorName>SYSLOG</ConnectorName>
<IsDefault>1</IsDefault>
<!-- select one of the following modes and remove the others -->
<ConnectionModes>
<ConnectionMode>
<InternalName>map</InternalName>
<DisplayName>Syslog:Map Output (appid)</DisplayName>
<Description>This mode outputs syslog data in "map" mode, meaning that header information is presented in special map variables.</Description>
<IsDefault>1</IsDefault>
<Properties>
<Property>
<Name>DataFormat</Name>
<Value>map</Value>
</Property>
<Property>
<Name>Applications</Name>
<Value>ABApplication</Value>
</Property>
<Property>
<Name>Stateful</Name>
<Value>false</Value>
</Property>
</Properties>
</ConnectionMode>
</ConnectionModes>
</ConnectionMethod>


Save the setting and close out that file again and re-run your tests; things should be a little better, at least up in that first 'Plug-In Tests' section.

In order to get any meaningful testing done on your end you'll need some sample data. Just in case you do not have some, I'll generate some that we can use for our generic collector and which we can then use for our mutual testing explorations:

{"s_AppId":"ABApplication","i_syslog_priority":"132","CONNECTION_METHOD":"SYSLOG","i_Hour":"15","i_RXBufferLength":"94","CONNECTION_MODE":"map","s_Process":null,"s_RV25":"1E873710-5E2D-1032-82AC-005056BE11EC","s_RV24":"5445E2E4-4CC4-1032-AE6D-005056BE11EC","i_Type":"2","i_Second":"35","s_RV23":"A47E3CC0-47FC-1032-A111-26FD14350435","s_RV22":"0B3DCFD7-37A3-1032-86F9-005056BE11EB","s_Version":"2011.1r5-201409150301-preview","s_RXBufferString":"Jan 04 15:52:35 EVTSRCBOX ABApplication Start {\"Application\":\"ABApplication\",\"Details\":\"Details go here\"}","s_RV21":"C350D010-3611-1032-8FAD-005056BE11EB","s_Body":"ABApplication Start {\"Application\":\"ABApplication\",\"Details\":\"Details go here\"}","s_chainId":"1417716103998","i_milliseconds":"1417729955000","s_raw_message2":"<132>Jan  4 15:52:35  EVTSRCBOX ABApplication Start {\"Application\":\"ABApplication\",\"Details\":\"Details go here\"}","s_MessageOriginatorPort":"56020","i_Minute":"52","s_Date":"Jan 04 15:52:35","i_TrustDeviceTime":"","i_DayOfMonth":"4","s_chainSequence":"0","i_Year":"2014","s_sha256Hash":"0d4c93ffac8cc4a38d2f94c48598eb1d654aaa02e334c1d2bab657f0fc405ce8","s_SyslogRelayIp":"10.10.10.1","s_MessageOriginatorHost":"EVTSRCBOX","s_Pid":null,"i_Month":"0","i_syslog_facility":"16","i_syslog_severity":"4"}
{"s_AppId":"ABApplication","i_syslog_priority":"131","CONNECTION_METHOD":"SYSLOG","i_Hour":"15","i_RXBufferLength":"113","CONNECTION_MODE":"map","s_Process":null,"s_RV25":"1E873710-5E2D-1032-82AF-005056BE11EC","s_RV24":"5445E2E4-4CC4-1032-AE6D-005056BE11EC","i_Type":"2","i_Second":"35","s_RV23":"A47E3CC0-47FC-1032-A111-26FD14350435","s_RV22":"0B3DCFD7-37A3-1032-86F9-005056BE11EB","s_Version":"2011.1r5-201409150301-preview","s_RXBufferString":"Jan 04 15:52:35 EVTSRCBOX ABApplication Error {\"Application\":\"ABApplication\",\"Error\":\"errName0\",\"Details\":\"Details go here\"}","s_RV21":"C350D010-3611-1032-8FAD-005056BE11EB","s_Body":"ABApplication Error {\"Application\":\"ABApplication\",\"Error\":\"errName0\",\"Details\":\"Details go here\"}","s_chainId":"1417716103998","i_milliseconds":"1417729955000","s_raw_message2":"<131>Jan 4 15:52:35 EVTSRCBOX ABApplication Error {\"Application\":\"ABApplication\",\"Error\":\"errName0\",\"Details\":\"Details go here\"}","s_MessageOriginatorPort":"56020","i_Minute":"52","s_Date":"Jan 04 15:52:35","i_TrustDeviceTime":"","i_DayOfMonth":"4","s_chainSequence":"1","i_Year":"2014","s_sha256Hash":"f7f46e55b79ece23225d00af9e91f7b6b42e58c238ee8a1445fe456be930338c","s_SyslogRelayIp":"10.10.10.1","s_MessageOriginatorHost":"EVTSRCBOX","s_Pid":null,"i_Month":"0","i_syslog_facility":"16","i_syslog_severity":"3"}
{"s_AppId":"ABApplication","i_syslog_priority":"132","CONNECTION_METHOD":"SYSLOG","i_Hour":"15","i_RXBufferLength":"93","CONNECTION_MODE":"map","s_Process":null,"s_RV25":"1E873710-5E2D-1032-82B1-005056BE11EC","s_RV24":"5445E2E4-4CC4-1032-AE6D-005056BE11EC","i_Type":"2","i_Second":"35","s_RV23":"A47E3CC0-47FC-1032-A111-26FD14350435","s_RV22":"0B3DCFD7-37A3-1032-86F9-005056BE11EB","s_Version":"2011.1r5-201409150301-preview","s_RXBufferString":"Jan 04 15:52:35 EVTSRCBOX ABApplication Stop {\"Application\":\"ABApplication\",\"Details\":\"Details go here\"}","s_RV21":"C350D010-3611-1032-8FAD-005056BE11EB","s_Body":"ABApplication Stop {\"Application\":\"ABApplication\",\"Details\":\"Details go here\"}","s_chainId":"1417716103998","i_milliseconds":"1417729955000","s_raw_message2":"<132>Jan 4 15:52:35 EVTSRCBOX ABApplication Stop {\"Application\":\"ABApplication\",\"Details\":\"Details go here\"}","s_MessageOriginatorPort":"56020","i_Minute":"52","s_Date":"Jan 04 15:52:35","i_TrustDeviceTime":"","i_DayOfMonth":"4","s_chainSequence":"2","i_Year":"2014","s_sha256Hash":"dba8ec06a4882548a15392dfe8b0024ed69fcf5a8133c0f70efaff770e67075c","s_SyslogRelayIp":"10.10.10.1","s_MessageOriginatorHost":"EVTSRCBOX","s_Pid":null,"i_Month":"0","i_syslog_facility":"16","i_syslog_severity":"4"}
{"s_AppId":"ABApplication","i_syslog_priority":"134","CONNECTION_METHOD":"SYSLOG","i_Hour":"15","i_RXBufferLength":"85","CONNECTION_MODE":"map","s_Process":null,"s_RV25":"1E873710-5E2D-1032-82B5-005056BE11EC","s_RV24":"5445E2E4-4CC4-1032-AE6D-005056BE11EC","i_Type":"2","i_Second":"35","s_RV23":"A47E3CC0-47FC-1032-A111-26FD14350435","s_RV22":"0B3DCFD7-37A3-1032-86F9-005056BE11EB","s_Version":"2011.1r5-201409150301-preview","s_RXBufferString":"Jan 04 15:52:35 EVTSRCBOX ABApplication Add {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\"}","s_RV21":"C350D010-3611-1032-8FAD-005056BE11EB","s_Body":"ABApplication Add {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\"}","s_chainId":"1417716103998","i_milliseconds":"1417729955000","s_raw_message2":"<134>Jan 4 15:52:35 EVTSRCBOX ABApplication Add {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\"}","s_MessageOriginatorPort":"56020","i_Minute":"52","s_Date":"Jan 04 15:52:35","i_TrustDeviceTime":"","i_DayOfMonth":"4","s_chainSequence":"3","i_Year":"2014","s_sha256Hash":"d15dd93de6dde4ef48f14857d6987c4f1d4b52aaa8ed1e4fd9ac7357ec08b00e","s_SyslogRelayIp":"10.10.10.1","s_MessageOriginatorHost":"EVTSRCBOX","s_Pid":null,"i_Month":"0","i_syslog_facility":"16","i_syslog_severity":"6"}
{"s_AppId":"ABApplication","i_syslog_priority":"134","CONNECTION_METHOD":"SYSLOG","i_Hour":"15","i_RXBufferLength":"88","CONNECTION_MODE":"map","s_Process":null,"s_RV25":"1E873710-5E2D-1032-82B7-005056BE11EC","s_RV24":"5445E2E4-4CC4-1032-AE6D-005056BE11EC","i_Type":"2","i_Second":"35","s_RV23":"A47E3CC0-47FC-1032-A111-26FD14350435","s_RV22":"0B3DCFD7-37A3-1032-86F9-005056BE11EB","s_Version":"2011.1r5-201409150301-preview","s_RXBufferString":"Jan 04 15:52:35 EVTSRCBOX ABApplication Modify {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\"}","s_RV21":"C350D010-3611-1032-8FAD-005056BE11EB","s_Body":"ABApplication Modify {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\"}","s_chainId":"1417716103998","i_milliseconds":"1417729955000","s_raw_message2":"<134>Jan 4 15:52:35 EVTSRCBOX ABApplication Modify {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\"}","s_MessageOriginatorPort":"56020","i_Minute":"52","s_Date":"Jan 04 15:52:35","i_TrustDeviceTime":"","i_DayOfMonth":"4","s_chainSequence":"4","i_Year":"2014","s_sha256Hash":"7ffc340bd3e617f99d301879ab0e7ecf91c524e7ff54479edc2210211b718520","s_SyslogRelayIp":"10.10.10.1","s_MessageOriginatorHost":"EVTSRCBOX","s_Pid":null,"i_Month":"0","i_syslog_facility":"16","i_syslog_severity":"6"}
{"s_AppId":"ABApplication","i_syslog_priority":"135","CONNECTION_METHOD":"SYSLOG","i_Hour":"15","i_RXBufferLength":"178","CONNECTION_MODE":"map","s_Process":null,"s_RV25":"1E873710-5E2D-1032-82C1-005056BE11EC","s_RV24":"5445E2E4-4CC4-1032-AE6D-005056BE11EC","i_Type":"2","i_Second":"35","s_RV23":"A47E3CC0-47FC-1032-A111-26FD14350435","s_RV22":"0B3DCFD7-37A3-1032-86F9-005056BE11EB","s_Version":"2011.1r5-201409150301-preview","s_RXBufferString":"Jan 04 15:52:35 EVTSRCBOX ABApplication Modify_Attribute {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\",\"AttributeName\":\"PhysicalDeliveryOfficeName\",\"AttributeValue\":\"6182 Fnwyp Park\"}","s_RV21":"C350D010-3611-1032-8FAD-005056BE11EB","s_Body":"ABApplication Modify_Attribute {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\",\"AttributeName\":\"PhysicalDeliveryOfficeName\",\"AttributeValue\":\"6182 Fnwyp Park\"}","s_chainId":"1417716103998","i_milliseconds":"1417729955000","s_raw_message2":"<135>Jan 4 15:52:35 EVTSRCBOX ABApplication Modify_Attribute {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\",\"AttributeName\":\"PhysicalDeliveryOfficeName\",\"AttributeValue\":\"6182 Fnwyp Park\"}","s_MessageOriginatorPort":"56020","i_Minute":"52","s_Date":"Jan 04 15:52:35","i_TrustDeviceTime":"","i_DayOfMonth":"4","s_chainSequence":"7","i_Year":"2014","s_sha256Hash":"02e965fb76f779618671e9ee415e10915a024af7fa7e3c5420c458133a4996ed","s_SyslogRelayIp":"10.10.10.1","s_MessageOriginatorHost":"EVTSRCBOX","s_Pid":null,"i_Month":"0","i_syslog_facility":"16","i_syslog_severity":"7"}
{"s_AppId":"ABApplication","i_syslog_priority":"135","CONNECTION_METHOD":"SYSLOG","i_Hour":"15","i_RXBufferLength":"220","CONNECTION_MODE":"map","s_Process":null,"s_RV25":"1E873710-5E2D-1032-82C3-005056BE11EC","s_RV24":"5445E2E4-4CC4-1032-AE6D-005056BE11EC","i_Type":"2","i_Second":"35","s_RV23":"A47E3CC0-47FC-1032-A111-26FD14350435","s_RV22":"0B3DCFD7-37A3-1032-86F9-005056BE11EB","s_Version":"2011.1r5-201409150301-preview","s_RXBufferString":"Jan 04 15:52:35 EVTSRCBOX ABApplication Modify_Attribute {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\",\"AttributeName\":\"PhysicalDeliveryOfficeName\",\"AttributeValue\":\"6182 Fnwyp Park\",\"AttributeOldValue\":\"Old value goes here\"}","s_RV21":"C350D010-3611-1032-8FAD-005056BE11EB","s_Body":"ABApplication Modify_Attribute {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\",\"AttributeName\":\"PhysicalDeliveryOfficeName\",\"AttributeValue\":\"6182 Fnwyp Park\",\"AttributeOldValue\":\"Old value goes here\"}","s_chainId":"1417716103998","i_milliseconds":"1417729955000","s_raw_message2":"<135>Jan 4 15:52:35 EVTSRCBOX ABApplication Modify_Attribute {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\",\"AttributeName\":\"PhysicalDeliveryOfficeName\",\"AttributeValue\":\"6182 Fnwyp Park\",\"AttributeOldValue\":\"Old value goes here\"}","s_MessageOriginatorPort":"56020","i_Minute":"52","s_Date":"Jan 04 15:52:35","i_TrustDeviceTime":"","i_DayOfMonth":"4","s_chainSequence":"8","i_Year":"2014","s_sha256Hash":"21892f60523f4294af01ebb6e4acdfaa4c5aa9733d5f3cfdf24c74b629b16661","s_SyslogRelayIp":"10.10.10.1","s_MessageOriginatorHost":"EVTSRCBOX","s_Pid":null,"i_Month":"0","i_syslog_facility":"16","i_syslog_severity":"7"}
{"s_AppId":"ABApplication","i_syslog_priority":"135","CONNECTION_METHOD":"SYSLOG","i_Hour":"15","i_RXBufferLength":"175","CONNECTION_MODE":"map","s_Process":null,"s_RV25":"1E873710-5E2D-1032-82C7-005056BE11EC","s_RV24":"5445E2E4-4CC4-1032-AE6D-005056BE11EC","i_Type":"2","i_Second":"35","s_RV23":"A47E3CC0-47FC-1032-A111-26FD14350435","s_RV22":"0B3DCFD7-37A3-1032-86F9-005056BE11EB","s_Version":"2011.1r5-201409150301-preview","s_RXBufferString":"Jan 04 15:52:35 EVTSRCBOX ABApplication Add_Attribute {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\",\"AttributeName\":\"PhysicalDeliveryOfficeName\",\"AttributeValue\":\"6182 Fnwyp Park\"}","s_RV21":"C350D010-3611-1032-8FAD-005056BE11EB","s_Body":"ABApplication Add_Attribute {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\",\"AttributeName\":\"PhysicalDeliveryOfficeName\",\"AttributeValue\":\"6182 Fnwyp Park\"}","s_chainId":"1417716103998","i_milliseconds":"1417729955000","s_raw_message2":"<135>Jan 4 15:52:35 EVTSRCBOX ABApplication Add_Attribute {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\",\"AttributeName\":\"PhysicalDeliveryOfficeName\",\"AttributeValue\":\"6182 Fnwyp Park\"}","s_MessageOriginatorPort":"56020","i_Minute":"52","s_Date":"Jan 04 15:52:35","i_TrustDeviceTime":"","i_DayOfMonth":"4","s_chainSequence":"9","i_Year":"2014","s_sha256Hash":"41907506215964aa05cfba4cd18c6d5411a914f42ab31445ce22e883b3538fff","s_SyslogRelayIp":"10.10.10.1","s_MessageOriginatorHost":"EVTSRCBOX","s_Pid":null,"i_Month":"0","i_syslog_facility":"16","i_syslog_severity":"7"}
{"s_AppId":"ABApplication","i_syslog_priority":"134","CONNECTION_METHOD":"SYSLOG","i_Hour":"15","i_RXBufferLength":"88","CONNECTION_MODE":"map","s_Process":null,"s_RV25":"1E873710-5E2D-1032-83DF-005056BE11EC","s_RV24":"5445E2E4-4CC4-1032-AE6D-005056BE11EC","i_Type":"2","i_Second":"31","s_RV23":"A47E3CC0-47FC-1032-A111-26FD14350435","s_RV22":"0B3DCFD7-37A3-1032-86F9-005056BE11EB","s_Version":"2011.1r5-201409150301-preview","s_RXBufferString":"Jan 04 15:56:31 EVTSRCBOX ABApplication Modify {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\"}","s_RV21":"C350D010-3611-1032-8FAD-005056BE11EB","s_Body":"ABApplication Modify {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\"}","s_chainId":"1417716103998","i_milliseconds":"1417729955000","s_raw_message2":"<134>Jan 4 15:56:31 EVTSRCBOX ABApplication Modify {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\"}","s_MessageOriginatorPort":"56020","i_Minute":"56","s_Date":"Jan 04 15:56:31","i_TrustDeviceTime":"","i_DayOfMonth":"4","s_chainSequence":"10","i_Year":"2014","s_sha256Hash":"fab66eab67b85974f33569895530dc149d0569cb85d6e3fdde70956ea433be30","s_SyslogRelayIp":"10.10.10.1","s_MessageOriginatorHost":"EVTSRCBOX","s_Pid":null,"i_Month":"0","i_syslog_facility":"16","i_syslog_severity":"6"}
{"s_AppId":"ABApplication","i_syslog_priority":"134","CONNECTION_METHOD":"SYSLOG","i_Hour":"15","i_RXBufferLength":"85","CONNECTION_MODE":"map","s_Process":null,"s_RV25":"1E873710-5E2D-1032-841A-005056BE11EC","s_RV24":"5445E2E4-4CC4-1032-AE6D-005056BE11EC","i_Type":"2","i_Second":"31","s_RV23":"A47E3CC0-47FC-1032-A111-26FD14350435","s_RV22":"0B3DCFD7-37A3-1032-86F9-005056BE11EB","s_Version":"2011.1r5-201409150301-preview","s_RXBufferString":"Jan 04 15:56:31 EVTSRCBOX ABApplication Add {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\"}","s_RV21":"C350D010-3611-1032-8FAD-005056BE11EB","s_Body":"ABApplication Add {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\"}","s_chainId":"1417716103998","i_milliseconds":"1417729955000","s_raw_message2":"<134>Jan 4 15:56:31 EVTSRCBOX ABApplication Add {\"workforceId\":\"t23456\",\"Initiator\":\"s12345\"}","s_MessageOriginatorPort":"56020","i_Minute":"56","s_Date":"Jan 04 15:56:31","i_TrustDeviceTime":"","i_DayOfMonth":"4","s_chainSequence":"12","i_Year":"2014","s_sha256Hash":"302f2fab0959049a65edba90bce7f827e8fe1c5c9311dbc934d576d21ff5ec95","s_SyslogRelayIp":"10.10.10.1","s_MessageOriginatorHost":"EVTSRCBOX","s_Pid":null,"i_Month":"0","i_syslog_facility":"16","i_syslog_severity":"6"}


The data above are just syslog data sent from a laptop to my Sentinel system; the AppID should match what we setup above ('ABApplication') and the data contained within the message has JSON within the syslog payload. If ever possible to write your own instrumentation for an application, sending data in JSON format over syslog is really, really, really nice, but more on that some other time. Take the data above, be sure each JSON event's line (starts with '{' and ends with a matching '}') is on exactly one line, and that there are no blank lines between each event. Be sure you are always using a valid text editor when manipulating files for use in IT, and that includes the SDK. Eclipse can open/modify this file, as can things like Notepad++ if you are stuck on windows; any editor on any other OS is probably fine, but some are better than others (beware of "helpful" line wrapping, invalid line-endings, etc.). The variety.dump file should be about 13,722 bytes in size with the data above pasted in. If off by more than a couple of bytes you may have something wrong. With things in place, re-run your tests and see if you get anything better from your test results. The bulk of the errors about data.in should now be gone, replaced with only valid errors about how the events are missing things, which is exactly what we expect at this point so we're well on our way.

Looking at the current errors, hopefully the problems are obvious. The 'Type-specific Tests' section is where most of our time will be spent at first to work though the NetIQ tests. In here is a lot of yellow with things like this:

11944942 - Validate XDAS_AE_CREATE_DATA_ITEM has required and recommended fields - No Events Matched the Match Condition. PluginTestObject.formattedResultMessage


The first field is the test number (should be unique), the second is a name that is often very helpful on its own, and the third is the detail from the test case success, warning, or failure. If the row is Yellow, that's basically a notification/warning and nothing more. For example, the line above means that none of the events generated by the collector were appropriate for the test. That may be a problem, but if the event source does not generate events matching that built-in test case then of course it is not a problem at all. These are those default XDAS Taxonomy test cases which verify that all fields are set properly based on the XDAS Taxonomy Name selected during parsing. If you know that your collector isn't using this particular type of event then move on.

At the bottom of the section are some legitimate errors which must be corrected to have a proper collector. These have names like, "All events should have ObserverEventTime, EventName, Message, Severity" which should be pretty obvious what is being checked. The details section gives you more detail, which is nice:

0 events passed out of 12 events matching the Match Condition:
Failed test on line 1: . From event undefined "ObserverEventTime" was empty or null but was expected to be not null. Test Result: fail
Failed test on line 10: . From event undefined "ObserverEventTime" was empty or null but was expected to be not null. Test Result: fail
Failed test on line 11: . From event undefined "ObserverEventTime" was empty or null but was expected to be not null. Test Result: fail
Failed test on line 12: . From event undefined "ObserverEventTime" was empty or null but was expected to be not null. Test Result: fail
Failed test on line 2: . From event undefined "ObserverEventTime" was empty or null but was expected to be not null. Test Result: fail
Failed test on line 3: . From event undefined "ObserverEventTime" was empty or null but was expected to be not null. Test Result: fail
Failed test on line 4: . From event undefined "ObserverEventTime" was empty or null but was expected to be not null. Test Result: fail
Failed test on line 5: . From event undefined "ObserverEventTime" was empty or null but was expected to be not null. Test Result: fail
Failed test on line 6: . From event undefined "ObserverEventTime" was empty or null but was expected to be not null. Test Result: fail
Failed test on line 7: . From event undefined "ObserverEventTime" was empty or null but was expected to be not null. Test Result: fail
Failed test on line 8: . From event undefined "ObserverEventTime" was empty or null but was expected to be not null. Test Result: fail
Failed test on line 9: . From event undefined "ObserverEventTime" was empty or null but was expected to be not null. Test Result: fail


Notice the 'Failed test on line #' text; the number reported is the line number of the input variety.dump file, so if you ever wonder which event is causing this, go look at the original file.

Since all events must have these attributes, the match condition is probably something like "if TRUE" (or equivalent) and either way the checks are failing on (in this case) the lack of the ObserverEventTime field. As a rule an event should have an ObserverEventTime just like they should an ObserverIP or ObserverHostName, and ReporterIP or ReporterHostName. These exist because Sentinel is here and reporting on its source of events, and the time of those events, so not-parsing these details out implies a problem with the event source or the system pulling from it (the collector). To fix this let's go into the release.js file to actually start doing some real parsing. Add the following lines in the parse() method and then try testing again:

//Regular expression for pulling apart the syslog message into its various parts.
this.evtRegex = new RegExp('^<(\\d+)>((\\S+)\\s+(\\d+)\\s+(\\d{2}:\\d{2}:\\d{2}))\\s+(\\S+)\\s+\\S+\\s(\\w+)\\s+(.+)$');
//Parsing of the raw data from the syslog connector.
this.evtAry = this.evtRegex.exec(this.s_raw_message2);
//Everything not statically in place goes into a JS object.
this.evtDetails = JSON.parse(this.evtAry[8]);
//Set the time and source according to the event source.
e.setObserverEventTime(DateTime.getDateTime(Number(this.i_milliseconds)));
//Set EventName
e.EventName = this.evtAry[7] + ' Event';
//And severity.
this.severity = 1;
//And a basic message. Without some combination of all of these, event parsing fails completely.
e.Message = 'Basic parsing of ' + e.EventName;


Disclaimer: the purpose of this code is to do something quickly to show how testing works, not to be great code, or smart code, or efficient code, or robust code.... just sample code.

All we have done here is set the basic fields required, plus a few others to help our first test work, for any event to be valid. Fixing just ObserverEventTime was nice, but ultimately the parsing fails down in the sendEvent() method in a way that cannot be found easily as it is in native code and therefore hitting the main while() loop's catch block and reporting 'Parsing failed.... blah blah blah'. Summary: Use the code above to set all of those fields so you have something worthwhile. When done, re-run your tests. What you should now see is a lot of green where before there was red, and a lot of details with 'All [#] Matched Events Passed' which is, of course, what we want. The applicable/matching NetIQ-shipped tests are all valid, so now we're on to our own tests.

Some simple examples that we'll create will be around the event name and the VendorEventCode, since the latter is likely to drive the former. For example, if the event from the application is type 'Add' then perhaps we want to have this be an 'Add Person' event. If it's a Modify, the same, 'Modify Person'. If it's an Add_Attribute or Modify_Attribute, though, those should simply become 'Add Attribute' and 'Modify Attribute', respectively. Finally, Start, Stop, and Error should continue through as 'Application Start', 'Application Stop', and 'Application Error', respectively. That last one, 'Application Error', should also be a Severity 5 event, where all other event types are Severity 1.

As you can see, the requirements above are not intense, or outrageous, and should be easily tested. These are exact rules that are driven by business requirements (event names and severities of those events) and so they should be reliable. There may be things in the Message field that are worth verifying, and there may be other fields worth checking, but to keep this example reasonably simple we'll just do the bits above. At this point we'll create our own test files within the SDK. There are not examples out yet, but I'll provide those. In fact, here is our template for everything which we'll put under the 2011.1/tests directory named 1-1-modify-person-base.test:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<Test Name="modify-person-base" Number="1" Variation="1">
<TestType>
collector-data
</TestType>
<MatchCondition>
<![CDATA[
Event.EventName == 'Modify Person'
]]>
</MatchCondition>
<TestCondition>
<![CDATA[
PluginTestObject.assertEquals('Severity', Event.Severity, '1')
&& PluginTestObject.assertEquals('ObserverEventTime', Event.ObserverEventTime, '1417729955000')
&& PluginTestObject.assertEquals('VendorEventCode', Event.VendorEventCode, 'Modify')
&& PluginTestObject.assertEquals('EventName', Event.EventName, 'Modify Person')
&& PluginTestObject.assertEquals('ObserverServiceName', Event.ObserverServiceName, 'ABApplication')
&& PluginTestObject.assertEquals('ObserverServiceComponent', Event.ObserverServiceComponent, 'LOCAL 0')
&& PluginTestObject.assertEquals('Vulnerability', Event.Vulnerability, '0')
]]>
</TestCondition>
<ResultMessage>
<![CDATA[
PluginTestObject.formattedResultMessage
]]>
</ResultMessage>
</Test>


Now some explanation of this text is certainly in order, as it is very specific and test-case-like. First, this is XML data, and within the XML is some ECMAScript/JavaScript within CDATA elements. If you've used ECMAScript in the SDK then that's probably enough to get you going, but there are specifics about this file that are particular and exact. The test files must be named something.test and are to be placed in the /path/to/your/vendor/application/2011.1/tests directory directly. The names before the .test extension must, I am told, match up with the Number, Variation, and test name within the XML (see the second line of XML above). In this case, that means that the name of the file is exactly this under the 2011.1/tests directory: 1-1-modify-person-base.test I have not yet ventured into tweaking Variation, but I believe the 'Number' property is made to correspond with Testopia test cases, and in particular those within NetIQ's Testopia system; as a result, the numbers you generate just need to be unique for you. All of the results of all test cases will show up in the third section of the test output, labeled '3. Release-specific Tests'.

Having taken the text above, created the 2011.1/tests/1-1-modify-person-base.test file, and pasted the data above within that file, you should now be able to run your test cases again and see a new line in section 3 as mentioned. That new line, if your data all match mine, will be the following yellow line:

11 - modify-person-base - No Events Matched the Match Condition. PluginTestObject.formattedResultMessage


In summary, our events are not setup properly for this new test; the first field is the number and iteration of the test case; the middle section is the name; the last bit there is static text from the test case, so we may want to do something more-sophisticated there than print 'PluginTestObject.formattedResultMessage'.

Since we have found a "bug" in our parse() method's logic, let's modify it a bit to try again, setting up all events to be named '<vendorcode> Person' instead of '<vendorcode> Event' ('Modify Person' instead of 'Modify Event' in this case):

//Regular expression for pulling apart the syslog message into its various parts.
this.evtRegex = new RegExp('^<(\\d+)>((\\S+)\\s+(\\d+)\\s+(\\d{2}:\\d{2}:\\d{2}))\\s+(\\S+)\\s+\\S+\\s(\\w+)\\s+(.+)$');
//Parsing of the raw data from the syslog connector.
this.evtAry = this.evtRegex.exec(this.s_raw_message2);
//Everything not statically in place goes into a JS object.
this.evtDetails = JSON.parse(this.evtAry[8]);
//Set the time and source according to the event source.
e.setObserverEventTime(DateTime.getDateTime(Number(this.i_milliseconds)));
//Set EventName
e.EventName = this.evtAry[7] + ' Person';
//And severity.
this.severity = 1;
//And a basic message. Without some combination of all of these, event parsing fails completely.
e.Message = 'Basic parsing of ' + e.EventName;


Our new detailed output from running our tests, now in red, should be the following:

0 events passed out of 2 events matching the Match Condition:
Failed test on line 11: . From event undefined "VendorEventCode" was "undefined" but was expected to be "Modify". Test Result: fail
Failed test on line 5: . From event undefined "VendorEventCode" was "undefined" but was expected to be "Modify". Test Result: fail


The important part here is that while things have gone from yellow to red, the reason is that before the test case did not match because the condition was wrong in that the event's EventName field was never 'Modify Person'. Now that the event matches, the test case attempts to verify things and errors stating that not everything is correct; it's a step forward, even though it feels at first like a step backward. Let's modify our parse() method logic from what we had before to instead be the following, basically adding some required fields (per our test case):

//Regular expression for pulling apart the syslog message into its various parts.
this.evtRegex = new RegExp('^<(\\d+)>((\\S+)\\s+(\\d+)\\s+(\\d{2}:\\d{2}:\\d{2}))\\s+(\\S+)\\s+\\S+\\s(\\w+)\\s+(.+)$');
//Parsing of the raw data from the syslog connector.
this.evtAry = this.evtRegex.exec(this.s_raw_message2);
//Everything not statically in place goes into a JS object.
this.evtDetails = JSON.parse(this.evtAry[8]);
//Set the time and source according to the event source.
e.setObserverEventTime(DateTime.getDateTime(Number(this.i_milliseconds)));
//Set EventName
e.EventName = this.evtAry[7] + ' Person';
//And severity.
this.severity = 1;
//And include the vendor event code
e.VendorEventCode = this.evtAry[7];
//And a basic message. Without some combination of all of these, event parsing fails completely.
e.Message = 'Basic parsing of ' + e.EventName;
//Set some value for Vulnerability
e.Vulnerability = 0;


There should be a nice change here, specifically the output message should be the following in green:

11 - modify-person-base - All 2 Matched Events Passed


This should lead to all kinds of warm, fuzzy feelings. We started with generic NetIQ test case failures and fixed our events so that those would work. Next we created our own test case based on business requirements, but it did not match events, so we fixed our parse() method logic. Once matching, our test case properly found a problem or two, which we also then fixed and now everything is testing successfully. In other words, we went through the process of getting our first test case created. Now we need more to handle the other business requirements.

The 'Add' event is basically the same as the 'Modify' event, but with 'Add' in place of 'Modify', so copy the current 1-1-modify-person-base.test file to 2-1-add-person-base.test and modify the strings accordingly. Don't cheat by looking at the code below until you have completed that, then see if things work. If not, try to fix it yourself to understand the messages coming from the SDK, then use the code below for an answer on how to build the test case; do not forget to update the XML portions of the file (not just the ECMAScript part):

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<Test Name="add-person-base" Number="2" Variation="1">
<TestType>
collector-data
</TestType>
<MatchCondition>
<![CDATA[
Event.EventName == 'Add Person'
]]>
</MatchCondition>
<TestCondition>
<![CDATA[
PluginTestObject.assertEquals('Severity', Event.Severity, '1')
&& PluginTestObject.assertEquals('ObserverEventTime', Event.ObserverEventTime, '1417729955000')
&& PluginTestObject.assertEquals('VendorEventCode', Event.VendorEventCode, 'Add')
&& PluginTestObject.assertEquals('EventName', Event.EventName, 'Add Person')
&& PluginTestObject.assertEquals('ObserverServiceName', Event.ObserverServiceName, 'ABApplication')
&& PluginTestObject.assertEquals('ObserverServiceComponent', Event.ObserverServiceComponent, 'LOCAL 0')
&& PluginTestObject.assertEquals('Vulnerability', Event.Vulnerability, '0')
]]>
</TestCondition>
<ResultMessage>
<![CDATA[
PluginTestObject.formattedResultMessage
]]>
</ResultMessage>
</Test>


Next we need those Start, Stop, and Error events. Those require the event name syntax to change; our current release.js file creates 'Start Person', etc. which is just silly. These are events indicating the start and stop of the event source itself, which could be useful information to detect when auditing changes happen (especially if unauthorized) or when auditing is failing due to an error. For this we need not only new test files for each of the three types of events, but we also need to update the release.js to set the EventName correctly for these types of events. Go ahead and try it out, but basically I will replace the single line to set the EventName with this little block of code:

//Set EventName
switch(this.evtAry[7]){
case 'Add':
case 'Modify':
e.EventName = this.evtAry[7] + ' Person';
break;
case 'Modify_Attribute':
e.EventName = 'Modify Attribute';
break;
case 'Add_Attribute':
e.EventName = 'Add Attribute';
break;
case 'Start':
case 'Stop':
case 'Error':
e.EventName = 'Application ' + this.evtAry[7];
break;
default:
//Unsupported event.
this.sendUnsupported();
return false;
}


I'll also create a 3-1-application-start-base.test file with the following contents:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<Test Name="application-start-base" Number="3" Variation="1">
<TestType>
collector-data
</TestType>
<MatchCondition>
<![CDATA[
Event.EventName == 'Application Start'
]]>
</MatchCondition>
<TestCondition>
<![CDATA[
PluginTestObject.assertEquals('Severity', Event.Severity, '1')
&& PluginTestObject.assertEquals('ObserverEventTime', Event.ObserverEventTime, '1417729955000')
&& PluginTestObject.assertEquals('VendorEventCode', Event.VendorEventCode, 'Start')
&& PluginTestObject.assertEquals('EventName', Event.EventName, 'Application Start')
&& PluginTestObject.assertEquals('ObserverServiceName', Event.ObserverServiceName, 'ABApplication')
&& PluginTestObject.assertEquals('ObserverServiceComponent', Event.ObserverServiceComponent, 'LOCAL 0')
&& PluginTestObject.assertEquals('Vulnerability', Event.Vulnerability, '0')
]]>
</TestCondition>
<ResultMessage>
<![CDATA[
PluginTestObject.formattedResultMessage
]]>
</ResultMessage>
</Test>


and a 4-1-application-stop-base.test with the following contents:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<Test Name="application-stop-base" Number="4" Variation="1">
<TestType>
collector-data
</TestType>
<MatchCondition>
<![CDATA[
Event.EventName == 'Application Stop'
]]>
</MatchCondition>
<TestCondition>
<![CDATA[
PluginTestObject.assertEquals('Severity', Event.Severity, '1')
&& PluginTestObject.assertEquals('ObserverEventTime', Event.ObserverEventTime, '1417729955000')
&& PluginTestObject.assertEquals('VendorEventCode', Event.VendorEventCode, 'Stop')
&& PluginTestObject.assertEquals('EventName', Event.EventName, 'Application Stop')
&& PluginTestObject.assertEquals('ObserverServiceName', Event.ObserverServiceName, 'ABApplication')
&& PluginTestObject.assertEquals('ObserverServiceComponent', Event.ObserverServiceComponent, 'LOCAL 0')
&& PluginTestObject.assertEquals('Vulnerability', Event.Vulnerability, '0')
]]>
</TestCondition>
<ResultMessage>
<![CDATA[
PluginTestObject.formattedResultMessage
]]>
</ResultMessage>
</Test>


and a 5-1-application-error-base.test with the following contents:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<Test Name="application-error-base" Number="5" Variation="1">
<TestType>
collector-data
</TestType>
<MatchCondition>
<![CDATA[
Event.EventName == 'Application Error'
]]>
</MatchCondition>
<TestCondition>
<![CDATA[
PluginTestObject.assertEquals('Severity', Event.Severity, '5')
&& PluginTestObject.assertEquals('ObserverEventTime', Event.ObserverEventTime, '1417729955000')
&& PluginTestObject.assertEquals('VendorEventCode', Event.VendorEventCode, 'Error')
&& PluginTestObject.assertEquals('EventName', Event.EventName, 'Application Error')
&& PluginTestObject.assertEquals('ObserverServiceName', Event.ObserverServiceName, 'ABApplication')
&& PluginTestObject.assertEquals('ObserverServiceComponent', Event.ObserverServiceComponent, 'LOCAL 0')
&& PluginTestObject.assertEquals('Vulnerability', Event.Vulnerability, '0')
]]>
</TestCondition>
<ResultMessage>
<![CDATA[
PluginTestObject.formattedResultMessage
]]>
</ResultMessage>
</Test>


Next we need to set the severity on an Error event appropriately in release.js; it is the only event required to be set to a severity of 5, so we'll change the one severity line in the parse() method to instead look like this:

  this.severity = ((e.EventName == 'Application Error') ? ('5') : ('1'));


Okay, let's test and see what breaks. If you have the same sample data, release.js, and test files as I do, nothing should break and we should now have five test cases reporting successful verification of parsed data. Finally we'll create two more test cases for the 'Add Attribute' and 'Modify Attribute' events. Purely out of laziness I'm going to implement new code in release.js to pull out the attribute name and values from our JSON-based object and store those data in TargetAttributeName and TargetAttributeValue, and the lazy part is that I'm going to use my existing switch/case statements to handle this. Replace the two relevant case statements and their commands with the following:

  case 'Modify_Attribute':
e.EventName = 'Modify Attribute';
e.TargetAttributeName = this.evtDetails.AttributeName;
e.TargetAttributeValue = this.evtDetails.AttributeValue;
break;
case 'Add_Attribute':
e.EventName = 'Add Attribute';
e.TargetAttributeName = this.evtDetails.AttributeName;
e.TargetAttributeValue = this.evtDetails.AttributeValue;
break;


I'll also create two new test case files with the following contents, respectively:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<Test Name="add-attribute-base" Number="6" Variation="1">
<TestType>
collector-data
</TestType>
<MatchCondition>
<![CDATA[
Event.EventName == 'Add Attribute'
]]>
</MatchCondition>
<TestCondition>
<![CDATA[
PluginTestObject.assertEquals('Severity', Event.Severity, '1')
&& PluginTestObject.assertEquals('ObserverEventTime', Event.ObserverEventTime, '1417729955000')
&& PluginTestObject.assertEquals('VendorEventCode', Event.VendorEventCode, 'Add_Attribute')
&& PluginTestObject.assertEquals('EventName', Event.EventName, 'Add Attribute')
&& PluginTestObject.assertEquals('ObserverServiceName', Event.ObserverServiceName, 'ABApplication')
&& PluginTestObject.assertEquals('ObserverServiceComponent', Event.ObserverServiceComponent, 'LOCAL 0')
&& PluginTestObject.assertEquals('Vulnerability', Event.Vulnerability, '0')
&& PluginTestObject.assertEquals('TargetAttributeName', Event.TargetAttributeName, 'PhysicalDeliveryOfficeName')
&& PluginTestObject.assertEquals('TargetAttributeValue', Event.TargetAttributeValue, '6182 Fnwyp Park')
]]>
</TestCondition>
<ResultMessage>
<![CDATA[
PluginTestObject.formattedResultMessage
]]>
</ResultMessage>
</Test>


and:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<Test Name="add-attribute-base" Number="7" Variation="1">
<TestType>
collector-data
</TestType>
<MatchCondition>
<![CDATA[
Event.EventName == 'Modify Attribute'
]]>
</MatchCondition>
<TestCondition>
<![CDATA[
PluginTestObject.assertEquals('Severity', Event.Severity, '1')
&& PluginTestObject.assertEquals('ObserverEventTime', Event.ObserverEventTime, '1417729955000')
&& PluginTestObject.assertEquals('VendorEventCode', Event.VendorEventCode, 'Modify_Attribute')
&& PluginTestObject.assertEquals('EventName', Event.EventName, 'Modify Attribute')
&& PluginTestObject.assertEquals('ObserverServiceName', Event.ObserverServiceName, 'ABApplication')
&& PluginTestObject.assertEquals('ObserverServiceComponent', Event.ObserverServiceComponent, 'LOCAL 0')
&& PluginTestObject.assertEquals('Vulnerability', Event.Vulnerability, '0')
&& PluginTestObject.assertEquals('TargetAttributeName', Event.TargetAttributeName, 'PhysicalDeliveryOfficeName')
&& PluginTestObject.assertEquals('TargetAttributeValue', Event.TargetAttributeValue, '6182 Fnwyp Park')
]]>
</TestCondition>
<ResultMessage>
<![CDATA[
PluginTestObject.formattedResultMessage
]]>
</ResultMessage>
</Test>


At this point we should have seven successful custom test cases, in addition to the NetIQ-shipped test cases. All of these test cases are pretty basic, but we need to start somewhere, and now any changes made in the future can have a very simple way to verify that old events are still being parsed as expected. How elaborate the test cases get is basically up to you. Assuming you control the input data you are able to even check things like we did in terms of ObserverEventTime, which would normally change with every event ever generated. It my be a good idea to have some of the fields be a bit more variable, or, when derived from one-another, have the test cases account for that. The Message field in particular can be hard because it often has variable data within but since you have ECMAScript at your disposal creating regular expressions or dynamically-generated patterns shouldn't be too hard if that adds value.

To add one more bit of value to the system, since we talked about XDAS Taxonomy earlier let's prove that a few more of the default NetIQ tests which are not currently matching will work with our taxonomy. Use the taxonomy.map sample found earlier in this article and replace the one in your collector with it, modifying the file in text mode. Next, add the following lines in your parse() method so that they fire for all events (perhaps after the e.Vulnerability variable is set):

//Set taxonomy for all events.
e.setTaxKey(this.evtAry[7]);


All that we are doing at this point is setting up taxonomy mappings, and then telling our collector to set some mappings to all of our types of events. The end result is that you'll actually get a bunch more errors. For some types of events you'll see that the NetIQ tests verify you have TargetUserDomain and TargetUserName; other types of events will need InitiatorUserDomain, InitiatorUserName, and TargetServiceName; still others will require SourceIP, SourceHostName, TargetIP, and TargetHostName. All of these things are required per the taxonomy of the events, so to have everything be completely proper you now know to pull in these other data as well. Without the tests built into the SDK, parsing would have worked, but some reports relying on these details may not have been as reliable as desired, and the events would obviously be less-complete and less-useful. Go ahead and fix the errors; no answers on this one, but you could have it done with junk data in about one minute, and with valid data in about ten.

As a last note, if you expand the 2011.1/tests/data directory you should see a file full of JSON with your parsed events for record keeping. Want to see how many times you made mistakes over the years, or want to see how you used to parse things? They're stored in there as part of this collector's build directory forever, so feel free to refer to them. If your filesystem gets a bit cluttered and the data are ancient, perhaps clean out those files, or maybe clean those out as a rule before committing everything to a version control system like git.

Happy Computing.

 

DISCLAIMER:

Some content on Community Tips & Information pages is not officially supported by Micro Focus. Please refer to our Terms of Use for more detail.
Top Contributors
Version history
Revision #:
1 of 1
Last update:
‎2015-01-28 20:51
Updated by:
 
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.