Highlighted
Absent Member.
Absent Member.
1811 views

Tools and tips to administer HP ArcSight CORR-based SIEM

THE FOLLOWING TEXT IS FOR SEARCH

Toolsandtipstoadminister HPArcSightCORR- basedSIEM (ESM6C) Samir Bennacer, Technical Solutions Consultant © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Agenda • Overview • Storage • Eventstorage • ResourceStorage • Archivestorage • Tools • ArcsightServices 2 © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.CORR -Engine:Overview © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.CORR-Engine:Overview CORR-Engine C Events Logger O C Logger Event Storage o ServerArcSight Event StoEngine R Store R m m - ESM E L Events InnoDB Manager n Storage g a MySQL y Engine i Resources n e and data r e 4 © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Storages © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Storages System storages Events storage Archives storage •Resources Events •Data files containing events •Trends and indexes of one day •Reports •Annotation •Rules •Active List and session List • … 6 © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Systemstorage InnodbStorageEngine All arcsight resourcearestored in Innodb Storage engine mysql>select TABLE_NAME, TABLE_TYPE, TABLE_SCHEMA, enginefrom information_schema.tables where engine like ‘InnoDB'; Viewingsize ofall thetables of innodb enginein CORR http://support.openview.hp.com/selfsolve/document/KM00238374 7 © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Eventstorage Arc logger storage engine All events arestored in LoggerStorage Engine mysql>select TABLE_NAME, TABLE_TYPE, TABLE_SCHEMA, enginefrom information_schema.tables where engine like 'ARC_LOGGER'; +------------+------------+--------------+------------+ |TABLE_NAME | TABLE_TYPE | TABLE_SCHEMA | engine | +------------+------------+--------------+------------+ |arc_event | BASE TABLE | arcsight | ARC_LOGGER | |events | BASE TABLE |arcsight | ARC_LOGGER | +------------+------------+--------------+------------+ 8 © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Eventstorage Event retention Retention period– Defined by max age or space. This defines the “on-line” events MRT (manager-receipt-time)based data retention (notend time)Oldest events will be overwritefirst(FIFO) Events pruningbased on theage of events Via managementconsole (web UI) 9 © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Eventstorage Configuring event retention 10 © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice. 10 HP ConfidentialStoragelayout Eventstoragelayout 1-Jul 2-Jul 3-Jul 4-Jul 5-Jul 6-Jul 7-Jul free 11© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Eventstorage Data storage architecture • “Chunk”containscompressedevents • Data file containsmultiplechunks File containingmultiple chunks:/opt/arcsight/logger/data/logger/ Arcsight_Data_ 12© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Eventstorage Database tables inpostgres • alg_storagevolume Contain volume storageconfiguration( file system type, size, path ) /opt/arcsight/logger/current/arcsight/bin/psqlrwdbweb-c "select* from alg_storagevolume;" • alg_storagegroup Contain the list of storagegroup and configuration( Retention) /opt/arcsight/logger/current/arcsight/bin/psqlrwdbweb-c "select* from alg_storagegroup;" • Data.file– contains info about all the files in /opt/arcsight/logger/data/logger/ • Data.chunk–contains info about all the chunks 13 © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Archivestorage Archives- Data filescontainingeventsofoneday,whichhavebeen copiedto thearchivelocation,withtwo additionalfiles containing metadatarelated to thesedata files andanother files containing annotation of oneday 14© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Storagelayout Eventstoragelayout Archivesize 1-Jul 6-Jul 2-Jul 5-Jul 3-Jul 4-Jul 4-Jul 3-Jul 5-Jul 6-Jul 2-Jul 7-Jul 1-Jul 0 20 40 15© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Archivestorage Therearetwo modes of archiving: 1. Scheduled - Runs on a daily basis, archiving the events from the day before. 2. Manual(user-driven) We recommend to usescheduled modeand usethe manualmodefor retryingan unsuccessfulscheduledone. 16© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Archivestorage Configuringarchive schedule Timeto start the archive operation for the current day's events as wellas any days manually marked for "retry": 17© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Archivestorage Viewing and working with archives Fromthe managerConsole we get two lists of archives: 1. Archives of events that are also still in active storage ("online"). 2. Archives of events thatare no longer in active storage , Deactivated ("offline") 18© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Archivestorage 1. ...Still in active storage • Eacharchive will representone day's worth of events • Thenumberofarchivesinthislist will bethenumberofdaysthatfitinthe configuredretention policy constrained by both a time-dimension and a space- dimension e.g. a retention policy of 30 days will have upto 30 items in thelist .. less ifthereis notenoughspace. • Foreachitem(archive) in thelist therewill beidentifying information:Date, Archive ID • Thedifferent states that these archives canbe in are Pending,In-progress, Archived, Not-Archived. 19© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Archivestorage 2. …no longer in active storage • Initially will be in theDeactivated state thusevents are not accessible. • Can be in one of the following states Deactivated, Activating, and Activated 20© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Archivestorage rwdb=# select * fromdata.file; fileid | filename | totalspace | storagegroupid | --------+-----------------------------------------------------------------+------------+---------------------- 10 | /opt/arcsight/logger/data/logger/Arcsight_Data_3 | 1073741824 | CLEAN 13 | /opt/arcsight/logger/data/logger/Arcsight_Data_7 | 1073741824 | 48518346341351424 11 | /opt/arcsight/logger/data/logger/Arcsight_Data_4 | 1073741824 | CLEAN 14 | /opt/arcsight/logger/data/archives/20110913/ArcSight_Data_1.dat |1073741824| ghost- 0504403158265495553-648518346341351424 21© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Settingstorage sizes © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Settingstorage sizes Questionyou shouldask • Which storage is full? • Whyit is full? • Do you have free disk space available to increase it? 23© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Archivestorage size set thefollowing propertyin the /opt/arcsight/logger/current/arcsight/logger/user/logger/logger .properties file: logger.archive.space.allocated-in-gb= is in GB. 24© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Systemstorage size 1-Makea backupcopyof /opt/arcsight/logger/data/mysql/my.cnf 2-Update the followingpropertyin my.cnf file: innodb_data_file_path =ibdata1:10M;ibdata2:1G:autoextend:max:G Makesureto includetheG after themax size (nospace).For example,if yourmax size is 300, the property should be: innodb_data_file_path =ibdata1:10M;ibdata2:1G:autoextend:max:300G 25© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Eventstorage size 1-Connectto postgresql /opt/arcsight/logger/current/arcsight/bin/psqlrwdb web 2- Update storage volume size and default storage group size by using these commands: updatealg_storagevolume set size=1073741824::bigint*; updatealg_storagegroup set size= 1073741824::bigint* whereisdefault=true; 3-Quit psql by runningthe command\q. 26© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Toolsand queries © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.ScripttogatherThreadDumpsandSession Waits Shell script to automates thegatheringof thefollowing information: Statusof services Thread Dumps, whichshow whatthe manager is processing Session Waits, whichshow whatthe database is processing Manager logs, which show anyerrors related to the Manager CORR-Enginelogs, which show anydatabase related errors http://support.openview.hp.com/selfsolve/document/KM00223046 28© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Show20 largest tables SELECTconcat(table_schema,'.',table_name) asDatabase_Tablename, table_rows as Rows, concat(round(data_length/(1024*1024),2),'M') DATA, concat(round(index_length/(1024*1024),2),'M')idx, concat(round((data_length+index_length)/(1024*1024),2),'M')total_size, round(index_length/data_length,2)idxfracFROM information_schema.TABLES where table_name like '%arc_%' order by data_length+index_length DESC limit 20; Weneedtheresult whentroubleshooting : 1-Performanceissue 2-Upgradeissue causedby size ofthesession list or Active list tables (NGS-7107,NGS-7108 ) 29© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Truncate Logically,TRUNCATE TABLEis equivalentto aDELETE statementthatdeletes all rows, butthereare practical differences under some circumstances. For an InnoDB table: Ifthereare no FOREIGNKEY constraints, InnoDB performsfasttruncationbydropping theoriginal table andcreatingan emptyonewith thesamedefinition, whichis much faster than deleting rows oneby one. Ifthereare any FOREIGNKEY constraints that referencethetable, InnoDB processes TRUNCATE TABLE by deleting rows oneby one 30© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Slowconsole login (bugNGS-4091) Truncate notification tables 1. Shutdown all arcsight services 2. Start mysqld service : /etc/init.d/arcsight_services start mysqld 3. changedirectory to: /opt/arcsight/logger/current/arcsight/bin 4. Issue: ./mysql -u arcsight -p 5. enter: 6. enter: use arcsight; 7. set foreign_key_checks=0; 8. truncate table arc_notification_history; 9. truncate table arc_notification_registry; 10. set foreign_key_checks=1; 11. start allthe services : /etc/init.d/arcsight_services start 31 © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Deletingmanyrowsfrom alargetable Ifyouaredeleting manyrows from a large table, you may exceedthelock table size for an InnoDB table. To avoid thisproblem, or simply to minimizethetimethatthe table remains locked, the following strategy (whichdoes notuse DELETE at all) might behelpful: 1. CREATE TABLE table_copy LIKE table; 2. UseRENAME TABLEto atomically movetheoriginal tableoutof thewayand renamethe copyto theoriginal name: RENAME TABLE tableTO table_old, table_copy TO table; 3. Drop the original table: DROP TABLEtable_old ; 32© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Ifyouarenotdeletingall rawsfromlargetable? 1. Select the rowsnotto be deleted into anemptytable thathas the same structure as theoriginal table: INSERT INTO table_copy SELECT * FROM table WHERE ... ; 2. UseRENAME TABLEto atomically movetheoriginal tableoutof thewayand renamethe copyto theoriginal name: RENAME TABLE tTO table_old, table_copy TO table; 3. Drop the original table: DROP TABLEtable_old; 33© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.innodb_buffer_pool_size • A static variable thatspecifies the size of thecachefor InnoDBdata and indexes. • theinnodb_buffer_pool_size variable cannotbe changeddynamically and requires a server restart • A larger buffer configured byinnodb_buffer_pool_size meansthereis less I/O neededto access data in tables. This is becausethe InnoDBstorage engine stores yourfrequently useddatainmemory. http://support.openview.hp.com/selfsolve/document/KM00407419 34© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Deleting session listentries Stepsto deleteentries froma Session list table 1. Connectto SSH usingarcsight user 2.cd /opt/arcsight/logger/current/arcsight/bin 3. ./mysql -u arcsight -p 4. use arcsight 5.select id fromarc_resource where name like '' andresource_type = 45; 6. selectDATA_table_id from arc_session_listwhereid =''; 7. delete from arc_SLD_ ; 35© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Exampleifyouneedtodeleteentries inactivelist 'ResourceAccess' select id from arc_resourcewherenamelike 'Resource Access' andresource_type = 45; 1. Connectto SSH usingarcsight user 2.cd /opt/arcsight/logger/current/arcsight/bin 3. ./mysql-uarcsight -p 4. use arcsight 5. select id from arc_resourcewherename like 'Resource Access' andresource_type = 45; id | ]I22H-Q4BABCFWxmR1pUJiA== 36© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Exampleifyouneedtodeleteentries inactivelist 'ResourceAccess' 6. selectDATA_table_id from arc_session_listwhere id = ']I22H- Q4BABCFWxmR1pUJiA=='; +---------------+ |DATA_table_id | +---------------+ |1QV652 | +---------------+ 7. delete from arc_SLD_1QV652 ; 37© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Exampleifyouneedtodeleteentries inactivelist 'ResourceAccess' Ifthey aredeleting manyrows from a large table, they may exceedthelock table size for an InnoDB table. To avoid thisproblem, or simply to minimizethetimethatthe table remains locked, the following strategy (whichdoes notuse DELETE at all) might behelpful: 1. CREATE TABLE arc_SLD_1QV652_copyLIKE arc_SLD_1QV652; 2.UseRENAME TABLEto atomically movetheoriginal tableoutof thewayand renamethe copyto theoriginal name: RENAMETABLEarc_SLD_1QV652TOarc_SLD_1QV652_old,arc_SLD_1QV652_copy TO arc_SLD_1QV652; 3. Drop the original table: DROP TABLE arc_SLD_1QV652_old; 38© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Eventdatasorting performance Problem: Excessive temporary file spaceused when sorting Event data– MySQL • Use only the portion of event field that is required – Use global/local variable andArcSight SUBSTRING on Event field • Power ofArcSightSUBSTRING function – Multi byte safe – Dynamically reduce temporary file space usage 39© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Eventdatasorting performance Createa globalvariable forDestination GeoCountryCode 40© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Eventdatasorting performance Createa global variable forDestination GeoCountryCode usingarcsight_substrfunction 41© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Eventdatasorting performance Usethevariable in queryorder/group by clause 42© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Eventdatasorting performance Performancewith queryusingevent field directly Sat Jun 2910:21:11PDT 2013 -rw-rw----. 1arcsight arcsight 23,534,392 Jun29 10:21 #sql_5f30_0.MYD Performanceafter using ArcSight substring Sat Jun 29 11:24:19 PDT 2013 -rw-rw----. 1arcsight arcsight 604,728 Jun29 11:24 #sql_5f30_0.MYD 43© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Resourcebackup Exportingsystem tables from ExpressCORR Engine http://support.openview.hp.com/selfsolve/document/KM00207745 ImportingSystem Tables from Express CORR Engine http://support.openview.hp.com/selfsolve/document/KM00465571 44© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Mysqldump Mysqldumpcommandcanbeusedto backupthetables data thatarenotexported using arcsight system export Example to backup Trend data : cd /opt/arcsight/logger/current/arcsight/bin ./mysqldump-uarcsight -p $(/opt/arcsight/logger/current/arcsight/bin/mysql-uarcsight -p-Darcsight -Bse "show tables like 'arc_trend_%'") > /tmp/trendsdump.out Toimportor restorethisdump : mysql-uarcsight -p < /tmp/trendsdump.out 45© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.ArcSightservices and monit © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.ArcSight services Singlescripttomanage allArcSightservices Controlprocess dependence and startup sequence Restart failed service UnifiedcontrolofallArcSightserviceswith/sbin/servicearcsight_services arcsight_services help arcsight_services [start | stop | status | …] [all| logger| manager | mysqld | …] 47© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Process managementworkflow (ESM6c) 48© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Logfiles Service logs /opt/arcsight/services/logs/arcsight_services.log MonitLogs /opt/arcsight/services/monit/data/monit.log Upstart log /etc/init/arcsight*conf 49© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Donotstopservices usingstopall Whensoppingall services do notuse: Arcsight_services stop all Whenyou stop usingthecommandall, mysql will stop ( faster ) before themanager , andmanger will still communicatewithmysqlserver . 50© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Recommendedorder Arcsight_services stoparcsight_web Arcsight_services stop manager Arcsight_services stop logger Arcsight_services stopmysqlor Arcsight_services stoppostgesql 51© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Arcsight services Singlescripttomanage allArcSightservices Controlprocess dependence and startup sequence Restart failed service UnifiedcontrolofallArcSightserviceswith/sbin/servicearcsight_services arcsight_services help arcsight_services [start | stop | status | …] [all| logger| manager | mysqld | …] 52© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Formore information Attendthese sessions 1319- Beyondthe ESM administrator guide 1188- ESM healthcheck 1145- Correlatingefficiently:tips,techniques,andtroubleshooting 1160- A look at the latest HP ArcSight ESM 1135- Advanced contentmanagementfor hierarchical ArcSight deployments 1205- How to stop using a USB stick for contentdistribution Yourfeedbackisimportanttous.Pleasetakeafewminutestocompletethesessionsurvey. 53© Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.Thankyou © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice. Security for the newreality © Copyright 2013 Hewlett-Packard Development Company,L.P. Theinformation contained herein is subject to change without notice.

Labels (1)
Tags (1)
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.