ALG_2015 Respected Contributor.
Respected Contributor.
507 views

Sitescope log size is increasing

Hi All,

We have observed that sitescope logs is consuming high. So please suggest how to control this.

Sis Version: v11.32

Below is the sample log file details:

SiteScope2017_09_14 ->Approx 20GB+ per day

SiteScope2017_09_14.v2->Approx 200MB+ per day

Please suggest can we fix the sitescope log file size.

We have below settings in master.config, also debug is not enabled. So what will be the probable cause.

maxAuxLogSize=10000000

_maxLogSize=16000000

 

Labels (1)
0 Likes
4 Replies
Micro Focus Expert
Micro Focus Expert

Re: Sitescope log size is increasing

Hi ALG_2015,

you cannot control the size of the daily logfiles, the size of them depends on the number and type of monitors.
The more monitors you have, the more entries are logged into the daily file in that very day.
The entries being logged can differ from monitor type to monitor type.
You can only control the number of logfiles kept by SiteScope (default is 40), via the entry in master.config
_dailyLogKeepDays=40
The number of days of log information to keep before deleting old logs. 

What's strange though is that the v2 logfile is a lot smaller than the other one,
it's always the other way around, as the v2 logfile contains more information per entry then the pre-v2 format.

Did you check what's in the logfile? Perhaps one monitor went crazy and runs every second instead of every horu or something "funny".

Is it just the file 09_14 (which shouldn't be used by SiteScope anymore, but today it should write to SiteScope2017_09_19*), or do you see this for every day?
BTW, I found one other configuration parameter from master.config
  _dailyLogTotalLimit=
  The maximum total size in bytes of primary log data before old logs are deleted.
and the information
..
In the SiteScope UI, <SS>/groups/master.config, or persistency configuration if the option to keep daily logs (_dailyLogKeepDays=) is set to a number and that number is not being adhered to, check the setting for the limit by size option (_dailyLogTotalLimit=). 
The daily log total limit by size option supersedes the number of days to keep. 
If both options are set, SiteScope will ignore the number of days to keep logs and only adhere to the size of logs to keep.
..
By default the entry
_dailyLogTotalLimit
doesn't exist or is set to
_dailyLogTotalLimit=

Greetings
Siggi

Customer Support
Micro Focus

If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
0 Likes
ALG_2015 Respected Contributor.
Respected Contributor.

Re: Sitescope log size is increasing

Hi Siggi,

Thanks for the reply.

We have below settings:

_dailyLogKeepDays=40
_dailyLogTotalLimit=

But we are facing this issue from last 4-5 days only, Dily sitescope log file size is increasing.

PFB inline comments for u r questions:

Did you check what's in the logfile? Perhaps one monitor went crazy and runs every second instead of every horu or something "funny".

=>No unfortunately size is too high so unable to check, if any workaround plz suggest. But yes we have already validated the monitoring frequency & all monitors having more than 5 min frequency.

Is it just the file 09_14 (which shouldn't be used by SiteScope anymore, but today it should write to SiteScope2017_09_19*), or do you see this for every day?

=>Everyday new file is created but from last 4-5 days log file size is drastically increases. Which were previously in under 400 MB per day but now its more than 20 GB per day.

Even we have removed around 50% monitors with deleting many old disabled CI's . Still issue is not resolved. Due to this many script monitors are suddenly failing & sending unecessory alerts.

0 Likes
Micro Focus Expert
Micro Focus Expert

Re: Sitescope log size is increasing

Hi ALG_2015,

that really strange!

..
We have below settings:
_dailyLogKeepDays=40
_dailyLogTotalLimit=
..
That's good, and is the default.

..
But we are facing this issue from last 4-5 days only, Daily sitescope log file size is increasing.
No unfortunately size is too high so unable to check, if any workaround plz suggest.
But yes we have already validated the monitoring frequency & all monitors having more than 5 min frequency.

Everyday new file is created but from last 4-5 days log file size is drastically increases.
Which were previously in under 400 MB per day but now its more than 20 GB per day.
Even we have removed around 50% monitors with deleting many old disabled CI's . Still issue is not resolved.
Due to this many script monitors are suddenly failing & sending unecessory alerts.
..

Really odd.
To check the file content, even if it's a huge file, try something trivial like the type command on Windows
 open a command window
 cd <SiteScope>\logs
 type SiteScope2017_09_14.log

I was able to take a peak at the largest files on my boxes (3 GB or so), soI'm confident that it will work for 20 GB as well.

If this shouldn't work, there are a lot of tools available for free in the Internet which allow you to split the file into smaller chunks,
so that you can open them. I think Notepad++ can work on files in anay size as well, you can also use 7zip to split the file, you can save it as a .zip or .7z format.
Simply use "Add to Archive", there is an option called "Split volume, bytes". Just select how large you want the chunks. (it offers 10 MB for example)

As soon as we know what takes up all the space, it should be possible to resolve it.
I assume that either one or more monitors went crazy ad fill the logs with duplicate entries, or some other part write crap into it, it might even be an external tool doing this for whatever reason.

Greetings
Siggi

Customer Support
Micro Focus

If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
0 Likes
Super Contributor.. Charles L. Super Contributor..
Super Contributor..

Re: Sitescope log size is increasing

Just a quick note, since I had similar issue in the past.

Check the groups folder and look for big dyn file(s) there. Then, use the monitor id to look for the monitor's name in the Persistency. You would then know which monitor actually the culprit.

We had several URL monitors with someone put /(.*)/s into the match content, result in the entire website big dumped into the Sitescope's daily log., and it took few 7 GB per log file per day, while normally, it was only around 600MB.

Hope it helps!

Cheers,

Charles L.

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.