How to create and manage Incidents in ServiceNow from Sentinel Alerts with Aegis

over 5 years ago
In previous posts I have shown ( here ) how to integrate Aegis and Sentinel - but why?  I often get asked what does Aegis actually do, its not an easy question to answer without example.  The short answer is - it can do almost anything - the long answer is explaining what that means!  Aegis is essentially an open automation framework, what it automates is completely up to (and limited by) you.

This post will show how Aegis can be used to create and manage Incidents in a Service Desk system, freeing up IT staff from doing a repetitive mundane task like opening tickets to actually working on IT problems.  Aegis workflows are iterative in nature and evolve over time, an early version of this workflow may just be opening a Service Desk Incident, later versions may be performing remedial action on common problems like resetting user passwords, freeing up disk space etc.  further freeing up IT staff to work on more interesting problems.

All Aegis workflows start off with a plan.  Sorry, all successful Aegis workflows start off with a plan  This plan is always best regardless of how simple, to be done in a flowchart style.  Once decisions need to be made in a workflow flowcharts are really important to avoid logic problems.  Workflow designers just need to translate the flowchart into a workflow - a good deal of the "what if?" questions should be answered during the flowchart.  Like - What if nobody is assigned the Incident after 30 minutes? What if the Incident is created outside business hours? etc.  Not all "What If" questions need to be answered in the first production iteration of the workflow - but understanding which questions aren't going to be addressed produces a much better workflow than not asking any questions.  Plan to be able to handle each "what if" scenario in future iterations and add them to your flowchart.  This puts you in charge, not wondering why your workflow isn't very successful.

Aegis workflows are almost always a collaborative affair.  This example for example will require possibly require somebody who works with Sentinel and somebody from the Service HelpDesk to understand how things are currently done, and few managers to decide and agree on how this process will work.  The flowchart is actually completely independent to Aegis so a workflow designer may not be present, although its always advisable to have somebody with clear logical thinking involved to ensure the flowchart makes sense.  This will normally mean the workflow will start off on a flip-chart and then more usually on a dry wipe board to allow for changes before perhaps going onto a flow chart designing piece of software after the design is agreed.

A flowchart leaves little ambiguity about what the process is designed to do.


So reading the plan - Workflow will start on a Sentinel Alert, then open a ticket in Service Now and then update the alert in Sentinel with the Incident number etc.  It next checks the severity of the alert - if its less than 4 end the workflow.  If its a higher severity we want to make sure that the Incident is assigned to somebody withing 30 minutes.  This is done by checking the Incident every 5 minutes.  If the Incident does get assigned then update the alert again and end the workflow.  If the Incident is not assigned after 30 minutes, run an escalation workflow (an external modular workflow).  Its kind of hard to follow workflow logic in a sentence.

Now onto workflow implementation!  I'll just go through the integration steps and assume you know how to perform other operations in Aegis (after reading my previous articles!).  When you look at the plan, there are only a couple of different interactions with Sentinel and ServiceNow.  For in dept on how to configure the Sentinel activities go here.

The first question is how to trigger the workflow.  Without an adapter we don't have an integrated way of polling for alerts in the background so we need some other way to get sentinel alerts into Aegis.  Once we have an alert we can just define an Aegis trigger based on the alert.  One method is to have a workflow which polls on interval for new alerts, not a favorite of mine but possible.  I'll refer to this post Triggering Aegis Workflows from Remote Systems for a more suitable alternative.  Sentinel provides some simple automation actions like sending an email when an event occurs and I will use this.  Basically we send an email to Aegis when an alert I want to send to Aegis occurs.  This automatically filters out alerts we don't want to automate from the Aegis side which is nice.

** Note - Due to the way Sentinel collapses alerts, it is best for the Aegis integration to modify the correlation rule generating the alert on the Sentinel side, so that a field like the event name includes a unique value to prevent alert collapsing.  I used the dt (event time) field, not quite unique but good enough for this scenario **

Here all I've done is copied the default Send Mail action in Sentinel and configured the action to send the email to a mailbox managed by Aegis.  Aegis will detect the email and generate an event in Aegis - easy!


The next bit isn't necessary but would be best practice and makes trigger configuration easier especially when more criteria is required in the trigger.  We will create a new Event Definition in Aegis so that any time an email with 'Sentinel Alert' in the subject field and from 'sentinel@sigea.moc' is detected, it is defined as a Sentinel Event.  Aegis uses events in its terminology, alerts or events on remote systems are treated as events in Aegis.


Now we can define a trigger for our workflow that simply says to trigger when one Sentinel Event occurs...


Before we can create an Incident in Service Now we have to get some information about the alert and be able to update the alert.  The email which comes from Sentinel comes from the event and not the alert, so there is no handy field in the email action which contains the alertid.  Instead we can do a search for the Alert in Sentinel based on the event name - remember above this should be a unique value so the result is accurate and yields a single result - otherwise it isn't going to work!

So this is the workflow so far - it extracts the event name from the triggering event, logs into Sentinel and performs a search for the alert (over a time range) and then gets the alert details which will be used in the ServiceNow Incident.  There is clearly a few extra steps here than not in the flowchart - this is normal - if you tried to document the manual steps required for each operation it would be a huge flowchart!


** Note - this has no error handling and assumes ideal conditions ... so add error handling! Example : What if no alert is found? **


So now we are onto the next step - now we need to create an Incident in ServiceNow based on the event.  I've chosen ServiceNow for this example as it has a demo available on the web which is usable so there is no smoke and mirrors here!  At the time of writing this article access to the demo is at :

USER_ID = "admin";
PASSWORD = "admin";

We use the built-in activity  'Call a Webservice' to create the incident - we will use this activity for each ServiceNow interaction.  This is the connection setup:


This will allow us to select one of the incident handling methods from ServiceNow.  To create an incident we use the Insert method:


The configuration of the 'insert' method contains a long list of values!  They are not all needed so you can set the 'parameter not used for most of them'.  I just set a couple of them, some containing details from the alert:

Active,  Category, Description, Priority, Severity, Short Description, State

The activity outputs the Incident Number as part of the XML result so this needs to be parsed out and immediately used in the ' Update Sentinel Alert activity to add a comment and set the status etc.

The only other operation in the workflow interacting with Service Now in this example is figuring out if the Incident has been assigned to somebody or not.  To do this we run the 'Call a Webservice' activity again this time using the 'get' method.  This just requires one input parameter which again can be parsed from the XML result of the create incident activity:


Then all you need to is compare the default assigned to value to check if its been assigned to a real user.

The rest of the workflow is ordinary Aegis workflow logic, a bit of parsing and conditional connectors etc.





How To-Best Practice
Comment List
Related Discussions