Re: Microsoft Exchange 2010 PowerShell connector - Java exception - Micro Focus Community

stefan.oancea · ‎2015-03-31

Dear All,

I am facing an issue with installing a connector for mailbox audit events in Exchange 2010 SP2 - the connector I am using is Microsoft PowerShell Connector. This is not my first installation for such a connector, but this time I am receiving an exception I don't know how to handle.

Scenario information:

-> I have a clustered Exchange implementation and I am installing the connector on the member holding the replicated mailbox database (the active one is on the other member, but I suspect this should not be an issue considering how clustering works for Exchange 2010)

-> The user I am installing the connector with is in the Local Administrators group on the Exchange Server and also in the Organization Management group in terms of Exchange Security Groups

-> The Windows Server is a 2008

-> I have enabled auditing on the Exchange server and it works

Behavior:

-> The installation goes smoothly, except for the step immediately following introducing the Server FQDN and the path to the PowerShell - after clicking Next it waits unusually long while validating parameters and in the end it prompts a java.net.SocketTimeoutException: Read timed out

-> It asks me if I still want to continue, I do and continue with the installation

-> The connector starts without any issues, it registers with the manager and starts sending Raw Events

-> However, in terms of information from the Exchange Server it does not retrieve anything: no logs, no mailboxes file in /current/user/agent/agentdata folder and no temporary log files in the same mentioned folder

A few important ERRORs I get in the agentsetup file immediately after setup (and that I can't find in normal setups) are below:

[2015-03-26 15:38:18,039][ERROR][default.com.arcsight.i.a.t][_apiVerifySimpleParameters]

AxisFault

faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException

faultSubcode:

faultString: java.net.SocketTimeoutException: Read timed out

faultActor:

faultNode:

faultDetail:

{http://xml.apache.org/axis/}stackTrace:java.net.SocketTimeoutException: Read timed out

[2015-03-26 15:38:18,039][ERROR][default.com.arcsight.i.a.t][verifyParameters]

java.rmi.RemoteException: ; nested exception is:

java.net.SocketTimeoutException: Read timed out

at com.arcsight.i.a.t.a(t.java:2427)

at com.arcsight.i.a.t.a(t.java:2900)

at com.arcsight.agentsetup.NGConnectorParameterPanelProcessor$1.run(NGConnectorParameterPanelProcessor.java:120)

at com.arcsight.wizard.n.a(n.java:343)

at com.arcsight.agentsetup.NGConnectorParameterPanelProcessor.processNext(NGConnectorParameterPanelProcessor.java:127)

at com.arcsight.wizard.e.d$0.g(d$0.java:61)

at com.arcsight.wizard.kb.process(kb.java:562)

at com.arcsight.wizard.swing.BaseWizardPanelImpl$6.run(BaseWizardPanelImpl.java:367)

I also get many of these afterwards:

[2015-03-26 15:39:34,416][ERROR][default.com.arcsight.i.a.t][ensureLoggedIn]
AxisFault
faultCode: {http://xml.apache.org/axis/}HTTP
faultSubcode:
faultString: (401)Unauthorized
faultActor:
faultNode:
faultDetail:
{}:return code: 401
<?xml version="1.0" encoding="utf-8"?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><soapenv:Body><soapenv:Fault><faultcode xmlns:ns1="http://xml.apache.org/axis/">ns1:Server.Unauthenticated</faultcode><faultstring>Not authenticated (method [getSupportedAPIVersion])</faultstring><detail><ns2:hostname xmlns:ns2="http://xml.apache.org/axis/">Exchange-MB03</ns2:hostname></detail></soapenv:Fault></soapenv:Body></soapenv:Envelope>
{http://xml.apache.org/axis/}HttpErrorCode:401

(401)Unauthorized
at org.apache.axis.transport.http.HTTPSender.readFromSocket(HTTPSender.java:744)

When I start the Windows service I get no further errors, not even in the connector logs (or at least I haven't identified any).

There is more information I can provide, however I don't want to load the post excessively; if anyone knows what exactly happens in the step where I get the Exception (when verifying the FQDN and powershell access) it would also be good in order to try and run the eventual commands directly in powershell and see the outcome - perhaps I will find the issue that way.

Many thanks,

Stefan

stefan.oancea · ‎2015-05-04

Hello again,

Since I was not able to find the root cause of the issue described (also tried with official HP support), what I did was the following:

1. Wrote a powershell script to read Exchange Mailbox Events audit data, parse that and write it into log files

2. I included the script in a Scheduled Task that runs periodically and that searches only for the latest audit events

3. I developed a Flex Connector to read the log files mentioned above and map all relevant fields to ArcSight event schema

If anyone else encounters this issue, I think this approach is a really good workaround.

All the best,

Stefan

Bechara · ‎2015-05-20

Hello Stefan,

i am facing the same problem,

can you share with me your flex ?

stefan.oancea · ‎2015-05-20

Hello Bechara,

Yes, I can certainly share the Flex with you - however, the Flex is designed especially for the file format I am formatting after extracting audit information from the Exchange Server. So I will also share with you the script used in order to extract the audit information from Exchange.

Exchange Script:

"

#Exchange Server FQDN

$strFQDN="PVS-EXCH-01.pvslab.local"

#File holding information about users to be monitored

$strFILE="C:\Users\so-utilizator03\Desktop\ScheduledTasks\mailboxes.csv"

#Current Date and Extension of each written file, highlighting date and time of generation

$date=Get-Date

$fileExtension="$($date.Day).$($date.Month).$($date.Year).$($date.Hour).$($date.Minute)"

#Temporary name of file being written

$strOutput="C:\Users\so-utilizator03\Desktop\ScheduledTasks\TestOutput\myRetrieveMailboxAuditLog_v3.1.$fileExtension.txt.temp"

#Generating Exchange Shell Session

$strURL="http://"+$strFQDN+"/PowerShell/"

$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri $strURL -Authentication Kerberos

Import-PSSession $Session

#Read mailboxes to be audited

$mailboxes = import-Csv -Path $strFILE

#For each mailbox check audit events

foreach ($mailbox in $mailboxes )

{

$startdate = (Get-Date).AddMinutes(-5)

$enddate = Get-Date

$userAlias = $mailbox.AliasName

$userDisplay = $mailbox.DisplayName

$info = $mailbox.info

#Search for logs from the last five minutes

$SearchResults = @(Search-MailboxAuditLog -Identity $userAlias -LogonTypes "Admin,Delegate,Owner" -StartDate $startdate -EndDate $enddate -ShowDetail)

#For each log, write a streamlined event into the temporary file

foreach ($auditLog in $SearchResults)

{

}

#If file was created, rename it in order to be processed

if (Test-Path $strOutput)

{

Rename-Item $strOutput "myRetrieveMailboxAuditLog_v3.1.$fileExtension.txt"

}

#Close session

Remove-PSSession $Session

"

Mailboxes File (mailboxes to be audited) example:

"

AliasName,DisplayName,Info

so-utilizator01,so-utilizator01,so-utilizator01@pvslab.local

"

Flex Connector configuration:

"

delimiter=|

trim.tokens=true

contains.empty.tokens=true

token.count=17

token[0].name=Operation

token[0].type=String

token[1].name=OperationResult

token[1].type=String

token[2].name=LogonType

token[2].type=String

token[3].name=FolderPathName

token[3].type=String

token[4].name=ClientIPAddress

token[4].type=IPAddress

token[5].name=ClientMachineName

token[5].type=String

token[6].name=ClientProcessName

token[6].type=String

token[7].name=MailboxOwnerUPN

token[7].type=String

token[8].name=DestMailboxOwnerSid

token[8].type=String

token[9].name=LogonUserDisplayName

token[9].type=String

token[10].name=SourceItems

token[10].type=String

token[11].name=SourceFolders

token[11].type=String

token[12].name=ItemSubject

token[12].type=String

token[13].name=OriginatingServer

token[13].type=String

token[14].name=MailboxResolvedOwnerName

token[14].type=String

token[15].name=LastAccessed

token[15].type=TimeStamp

token[15].format=MM/dd/yyyy HH\:mm\:ss

token[16].name=SourceItemSubjectsList

token[16].type=String

event.deviceAction=Operation

event.eventOutcome=OperationResult

event.deviceCustomString1=LogonType

event.deviceCustomString1Label=__stringConstant("LogonType")

event.oldFilePath=FolderPathName

event.sourceAddress=ClientIPAddress

event.sourceHostName=ClientMachineName

event.deviceProcessName=ClientProcessName

event.destinationUserName=MailboxOwnerUPN

event.destinationUserId=DestMailboxOwnerSid

event.sourceUserName=LogonUserDisplayName

event.deviceCustomString2=SourceItems

event.deviceCustomString2Label=__stringConstant("MailboxItems")

event.deviceCustomString4=SourceFolders

event.deviceCustomString4Label=__stringConstant("SourceFolders")

event.deviceCustomString3=ItemSubject

event.deviceCustomString3Label=__stringConstant("ItemSubject")

event.deviceHostName=OriginatingServer

event.deviceCustomString5=MailboxResolvedOwnerName

event.deviceCustomString5Label=__stringConstant("OwnerName")

event.deviceReceiptTime=LastAccessed

event.name=__concatenate("Exchange Mailbox Audit - ",Operation)

event.deviceCustomString6=SourceItemSubjectsList

event.deviceCustomString6Label=__stringConstant("SourceSubjectsList")

event.deviceProduct=__stringConstant("Exchange 2010")

event.deviceVendor=__getVendor("Microsoft")

event.deviceSeverity=Operation

severity.map.low.if.deviceSeverity=Copy,Create

severity.map.medium.if.deviceSeverity=FolderBind,MessageBind,Move,MoveToDeletedItems

severity.map.high.if.deviceSeverity=SoftDelete,Update

severity.map.veryhigh.if.deviceSeverity=HardDelete,SendAs,SendOnBehalf

"

I do believe this covers quite everything in order to make it work - I have also added a few comments in the script so it is easy to understand/adapt. The powershell script searches for logs from the last 5 minutes, so make sure you set the Windows Scheduled Task to run every five minutes as well; and of course, change the paths in the script with you own. The Flex Connector is a multiple folder fixed string flex.

If you will see duplicate events from time to time, it is normal - all SIEM solutions seem to have this issue with Exchange 2010 Audit Events. I found out why as well - the Windows Powershell command which they (as well as my script) probably use is "Search-MailboxAuditLog" - no matter what window of time you assign to it, it will also search for entries from 10-15 minutes earlier than your window interval (so it will get some duplicates between two/three windows). If you want to minimize this effect, just make the window longer (so instead of 5 minutes use 30 minutes for both the script and the scheduled job) so that you will have less duplicates (you will still get some though).

Good luck,

Stefan

Bechara · ‎2015-05-24

Thank you Stefan for the script, but I can see that you are using Search-MailboxAuditLog, does this get admin logs?

I want to audit when and administrator create a new user, or change anything on the userMailbox,

does your script monitor those events?

stefan.oancea · ‎2015-05-25

Hello,

My script (as well as the default PowerShell Connector from ArcSight as far as I know) deal with Mailbox Audit Events - that is basically when UserA accesses UserB's mailbox, and what UserA is doing in there (delete e-mail, send on behalf of, move e-mail and so on). These are some special kind of logs which Microsoft starting from Exchange Server 2010 records in each user's mailbox.

If you need administrative activity on the Exchange Server I think you have to search for some other connector/Exchange logs.

All the best,

Stefan

mustapha_arakji · ‎2015-06-22

Thank you Stefan,

We used your code (Me and my colleague Bec A R) to build our own flex for thid, I hope the support could fix the main connector to save us some time

Mustapha

mwhit720 · ‎2016-06-15

Hello Mustapha,

Do you mind sharing the admin flex you built for Powershell?

alexandru.truta · ‎2016-12-12

Hello Marlon,

I've used the same approach as Stefan did for the Admin Audit Logs, and below you can find the configuration files.

The schedule task for the script is set to run every 1 hour.

Also, I'm successfully using a Multiple Folder File Flex Connector that includes both Stefan's parser and mine, with a minor change in wildcard properties from agent.properties.

PowerShell script:

#Exchange Server FQDN

$strFQDN="exchange.pvslab.local"

#Current Date and Extension of each written file, highlighting date and time of generation

$date=Get-Date

$fileExtension="$($date.Day).$($date.Month).$($date.Year).$($date.Hour).$($date.Minute)"

#Temporary name of file being written

$strOutput="C:\Users\adminAT\Desktop\Exchange_AdminAuditLog_Flex\myRetrieveAdminAuditLog_v1.$fileExtension.txt.temp"

#Generating Exchange Shell Session

$strURL="http://"+$strFQDN+"/PowerShell/"

$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri $strURL -Authentication Kerberos

Import-PSSession $Session

#Search Admin Audit Logs

$startdate = (Get-Date).AddMinutes(-1440)

$SearchResults = @(Search-AdminAuditLog -StartDate $startdate)

#For each log, write a streamlined event into the temporary file

foreach ($auditLog in $SearchResults)

{

}

#If file was created, rename it in order to be processed

if (Test-Path $strOutput)

{

Rename-Item $strOutput "myRetrieveAdminAuditLog_v1.$fileExtension.txt"

}

#Close session

Remove-PSSession $Session

FlexConnector configuration:

delimiter=|

trim.tokens=true

contains.empty.tokens=true

token.count=13

token[0].name=RunspaceId

token[0].type=String

token[1].name=ObjectModified

token[1].type=String

token[2].name=CmdletName

token[2].type=String

token[3].name=CmdletParameters

token[3].type=String

token[4].name=ModifiedProperties

token[4].type=String

token[5].name=Caller

token[5].type=String

token[6].name=Succeeded

token[6].type=String

token[7].name=Error

token[7].type=String

token[8].name=RunDate

token[8].type=TimeStamp

token[8].format=MM/dd/yyyy HH\:mm\:ss

token[9].name=OriginatingServer

token[9].type=String

token[10].name=Identity

token[10].type=String

token[11].name=IsValid

token[11].type=String

token[12].name=ObjectState

token[12].type=String

event.deviceExternalId=RunspaceId

event.deviceCustomString1=ObjectModified

event.deviceCustomString1Label=__stringConstant("ObjectModified")

event.deviceAction=CmdletName

event.deviceCustomString2=CmdletParameters

event.deviceCustomString2Label=__stringConstant("Parameters")

event.deviceCustomString3=ModifiedProperties

event.deviceCustomString3Label=__stringConstant("Properties")

event.sourceUserName=Caller

event.eventOutcome=Succeeded

event.deviceCustomString4=Error

event.deviceCustomString4Label=__stringConstant("Error")

event.deviceReceiptTime=RunDate

event.deviceHostName=OriginatingServer

event.externalId=Identity

event.deviceCustomString5=IsValid

event.deviceCustomString5Label=__stringConstant("IsValid")

event.deviceCustomString6=ObjectState

event.deviceCustomString6Label=__stringConstant("ObjectState")

event.name=__concatenate("Exchange Admin Audit - ",Caller)

event.deviceProduct=__stringConstant("Exchange 2010")

event.deviceVendor=__getVendor("Microsoft")

event.deviceSeverity=CmdletName

severity.map.low.if.deviceSeverity=Set-CalendarNotification,Set-CalendarProcessing

severity.map.medium.if.deviceSeverity=Set-MailboxCalendarFolder

severity.map.high.if.deviceSeverity=Set-Mailbox,Set-MailboxFolderPermission,Add-MailboxFolderPermission,Remove-MailboxFolderPermission

severity.map.veryhigh.if.deviceSeverity=Set-MailboxPermission,Add-MailboxPermission,Remove-MailboxPermission

All the best,

Alexandru

irfan_ansari017 · ‎2017-02-19

Good Day,

What user permissions do we need to execute this script.

Regards,

Irfan

stefan.oancea · ‎2017-02-20

Hello Irfan,

From what I recall, the user had to be part of the "Records Management" group: Records Management: Exchange 2013 Help .

Also, executing Powershell scripts had to be allowed as described in the connector configuration guide here:

I don't have the environment at hand anymore, but perhaps can check in his current implementation and let us know the AD Groups the user is part of.

All the best,

Stefan

alexandru.truta · ‎2017-02-21

Hello Irfan,

As you can see in the scripts, after creating the connection strings, the functions (named cmdlets by Microsoft) used to retrieve the audit logs are: Search-AdminAuditLog (https://technet.microsoft.com/en-us/library/ff459250(v=exchg.141).aspx ) and Search-MailboxAuditLog (Search-MailboxAuditLog).

In my test environment I'm using a domain admin account, but as you can find in the articles above, a user included in Organization Management and Record Management groups should be able to read the relevant audit logs.

Regards,

Alexandru

Microsoft Exchange 2010 PowerShell connector - Java exception

Smart Connector