Highlighted
Super Contributor.
Super Contributor.
585 views

Need Metric Documentation for all Monitors Types in the Daily Logs

We are forwarding all logs to Splunk. Users would like to know what each metrics values represents w/o having to login to SS and compare. I think we should have documentation defining this.

Labels (2)
0 Likes
10 Replies
Highlighted
Micro Focus Expert
Micro Focus Expert

Please clarify your requirements regarding log collection and streaming to Splunk. Are you referring to CDF service logs or Suite Application logs?

0 Likes
Highlighted
Super Contributor.
Super Contributor.

I'm referring to SiteScope Daily Logs containing all monitor run statistics.

0 Likes
Highlighted
Micro Focus Expert
Micro Focus Expert

Please indicate which suite(s) you are referring to. Also clarify use of SiteScope when logs are streamed to OBA or ELK/Kibana

0 Likes
Highlighted
Super Contributor.
Super Contributor.

Not sure what you mean by "indicate which suite(s)". So Splunk is our reporting tool and if teams want to leverage SS monitor run data, they need to know what metric each value represents. Is there documentation to tell me what nonlabelled metrics mean? Some metrics are labeled such as with the cpu monitor: "avg", "cpu1" or "cpu2", but there are some values that are not labeled and we have to reference SiteScope to determine that. Can there be documentation created to provide these details?

 

 

0 Likes
Highlighted
Contributor.. Contributor..
Contributor..

Hi @ADPRJ 

I'm looking also at using the log files with Splunk, so I was wondering if you managed to find any documentation regarding the log files?

 

Thanks!

0 Likes
Highlighted
Micro Focus Expert
Micro Focus Expert

Hi csaavedra,

in the past we had the log file layout document, but I cannot find this anymore 😞
In the 11.10 Online Help I was aboe to find
..
SiteScope Log File Columns

When SiteScope runs a monitor instruction to test the availability of components in the infrastructure, the monitor results are written to data log files. The first six columns of each log entry in a SiteScope monitor data log are the same for each monitor type. After the first six columns of each log entry, the content of each column is specific for each monitor type.

The following table describes the content of these columns. The columns in each log file are written as tab-delimited text.
Column Data in Column
1 Time and date the sample was recorded.
2 Category (for example, good, error, warning, nodata).
3 Monitor group name where the monitor defined (also called ownerID).
4 Monitor title text.
5 stateString (this is the status string that shows up on the Group details page).
6 id:sample number (a unique ID for this monitor where group + id is a unique key for a monitor). The sample number is a unique sample number for that monitor.
..

While I haven't found it documented anywhere, I found another way of getting to the information:
decode the v2 logfiles with thelp of DailyLogPrinter.bat

c:\sitescope\bin\DailyLogPrinter.bat -sourceFile C:\SiteScope\logs\SiteScope2020_09_15.v2.log -targetFile C:\SiteScope2020_09_15.v2.txt

one line from C:\SiteScope\logs\SiteScope2020_09_15.v2.log

00:00:06 09/15/2020 name#:#DB Query Oracle sov02bac14 BAC14MGMT external#:#e14690ca-bc71-48f8-a3ac-38751598b977 bsm#:#1000086 targets#:#sov02bac14.swinfra.net: status#:#good state#:#H4sIAAAAAAAAADNQKE5N1lEwVCjKL9dRKM4vMzBKSkw2NAEKmVqaGgAAqOSbyh8AAAA= metrics#:#H4sIAAAAAAAAAGWMQQ6AIAwEv9KED7REDsb4GEQSSYAaKPJ90YMXb7s7ky3c8g5SwgkSklerQrWo4muLAo5jSxkICnegwSpfqDfraPpJ+pPIzOY94V6fOqLjLD4LJCvuGNuYqlhpD9eIN/a/GnqFAAAA type#:#SQL Query unique#:#201722764

and the corresponding line from C:\SiteScope2020_09_15.v2.txt

00:00:06 09/15/2020 name#:#DB Query Oracle sov02bac14 BAC14MGMT external#:#e14690ca-bc71-48f8-a3ac-38751598b977 bsm#:#1000086 targets#:#sov02bac14.swinfra.net: status#:#good state#:#0 sec, 1 row, sov02bac14, 15950 metrics#:#round trip time#=#0#;#result column 1 row 1#=#sov02bac14#;#result column 2 row 1#=#15950#;#rows#=#1#;#content match#=##;#status#=#200 type#:#SQL Query unique#:#201722764

so you'll find the date, time, monitor name, and all metrics, in this case
#metrics#:
#round trip time#=#0#
#result column 1 row 1#=#sov02bac14#
#result column 2 row 1#=#15950#
#rows#=#1#
#content match#=##
#status#=#200

another example:
name#:#CPU sov02bac17.swinfra.net
metrics#:
#utilization#=#2#
#utilization cpu # 1#=#4#
#utilization cpu # 2#=#0

or
name#:#DynDisk sov02bac13.swinfra.net
metrics#:
#counters in error#=#0#
#Disk/File System/[C]/percent full#=#84.688#
#Disk/File System/[HarddiskVolume1]/percent full#=#74.499

Greetings
Siggi

Customer Support
Micro Focus

If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
0 Likes
Highlighted
Contributor.. Contributor..
Contributor..

Hi @Siggi Gladitsch 

Thanks a lot for your answer.

I have been trying to get more details about the files directly with the MF support but it has been unsuccessful.

I'm running  SiteScope on a Linux box and the DailyLogPrinter.bat or an equivalent doesn't seem to be present.

Do you know if there is another way to decrypt the V2 files?

Also do you mind to give me the contents of the batch file, maybe I can try to create a shell script with the java calls that are being done there.

Thanks again for your help!

0 Likes
Highlighted
Micro Focus Expert
Micro Focus Expert

Hi csaavedra,

I'm not aware of any other way to decode the logs.
I don't see any reason not to share the .bat file content from SiteScope 11.92 / 2020.05 on Windows with you:
*****
@echo off

set SCRIPT_PATH=%~dp0

%~d0
cd %SCRIPT_PATH%

set CLASS_PATH=%CLASS_PATH%;..\WEB-INF\lib\ss_platform.jar;..\WEB-INF\lib\log4j.jar;..\WEB-INF\lib\commons-codec.jar;..\WEB-INF\lib\javacore.jar;

set JAVA_LOCATION=..\java\bin
if exist ..\..\..\java\bin\java.exe set JAVA_LOCATION=..\..\..\java\bin
if exist ..\..\..\..\java\bin\java.exe set JAVA_LOCATION=..\..\..\..\java\bin

set MAIN_CLASS=com.mercury.sitescope.platform.log.printer.DailyLogPrinter

%JAVA_LOCATION%\java.exe -Xmx512M -classpath %CLASS_PATH% %MAIN_CLASS% %*
*****

The key seems ot be that you need to be able to call
  com.mercury.sitescope.platform.log.printer.DailyLogPrinter
residing in <SiteScope>\WEB-INF\lib\ss_platform.jar
asnd feeding the command line parameters to it.

Greetings
Siggi

Customer Support
Micro Focus

If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
0 Likes
Highlighted
Contributor.. Contributor..
Contributor..

Hi @Siggi Gladitsch 

Thanks so much again.

I managed to create a shell script and is working fine, I can decrypt the data on the V2 log files.

Below my code in case someone else in the community needs it

I put the script at  <SiteScope root directory>/bin/DailyLogPrinter.sh

lib_path=/opt/HP/SiteScope/WEB-INF/lib
class_path=$lib_path/ss_platform.jar:$lib_path/ss_infra.jar:$lib_path/log4j.jar:$lib_path/commons-codec.jar:$lib_path/javacore.jar

export class_path
java_location=../java/bin
if [ -f /opt/OV/nonOV/jre/b/bin/java ]; then
java_location=/opt/OV/nonOV/jre/b/bin
fi
if [ -f ../../../java/bin/java ]; then
java_location=../../../java/bin
fi
if [ -f ../../../../java/bin/java ]; then
java_location=../../../../java/bin
fi

main_class=com.mercury.sitescope.platform.log.printer.DailyLogPrinter

$java_location/java -Xmx512M -classpath $class_path $main_class "$@"

 

[xxxx@xxxxxx bin]$ ./DailyLogPrinter.sh


Usage: DailyLogDumper.bat "-sourceFile source_log_file-targetFile target_file
   -sourceFile is a mandatory parameter specifies the log file name which will be parsed.
-targetFile is a mandatory parameter specifies the path to the file where we want to save decompressed data.
 Example:
       DailyLogDumper.bat "-sourceFile SiteScope2012_03_12.v2.log -targetFile c://temp//example.txt

 

0 Likes
Highlighted
Micro Focus Expert
Micro Focus Expert

Hi csaavedra,

thanks a lot for providing a working script.

I created KM03719127 to document your script, so that others can easily find and use it.

Greetings
Siggi

Customer Support
Micro Focus

If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.