Having problems with your account or logging in?
A lot of changes are happening in the community right now. Some may affect you. READ MORE HERE
Outstanding Contributor.. mustapha_arakji Outstanding Contributor..
Outstanding Contributor..
225 views

run Report on old Data

I have a requirement to generate a Report on old data, for a whole month. What is the best approach for this? taking into consideration that I didn't have queries and trends created by that time.

Mustapha
Labels (2)
Tags (1)
5 Replies
Gayan Acclaimed Contributor.
Acclaimed Contributor.

Re: run Report on old Data

Hi Musthapa,

I think better way is try to generate report through logger.

Mr
0 Likes
pbrettle Acclaimed Contributor.
Acclaimed Contributor.

Re: run Report on old Data

I think Gayan is correct. Logger will be a better place to start with this type of report rather than trying with ESM.

Longer ER and more detailed reports are absolutely possible with ESM, but they will take time to product and finish. So make sure you follow these points:

1) build and test the report on a much smaller data set. For example do it only over a few hours or maybe even a 30 minute interval. Then test it and make sure you are ok with it. This is simple and easy to do and won't impact much at all.

2) when you want to run the report, schedule it to run at a low EPS time, such as midnight or something. It will be faster and easier based on low user interaction and EPS.

3) keep the tables small and simple. More advanced formatting and queries will clearlt take more time.

4) keep the volume of data going into the report small. Reports are great for summaries, not for millions of events. So consider what the data is and how you want to summarize it.

5) filter, filter and filter again. Keep the volume of events down and try to think about things laterally. What I mean is why do 3 Windows events when you can really dig into just one of them and build the report from there. Sounds simple I know, but you will be amazed on what can be reduced.

6) efficient queries are the key. Obvious reasons I know. But judicious uses of AND massively improves the logic rather than OR. And try to think about how the query works too. And avoid complex and time consuming operators like Contains etc.

hope this Helps.

Outstanding Contributor.. mustapha_arakji Outstanding Contributor..
Outstanding Contributor..

Re: run Report on old Data

Thanks for the advise.

Logger seems to be a good choice too, unfortunatly in this case, there's no logger.

Mustapha
0 Likes
stefan.oancea Outstanding Contributor.
Outstanding Contributor.

Re: run Report on old Data

Hello Mustapha,

I too was faced with a similar request - it was about reporting on FW logs for the past 3 months . So quite a lot of time and a lot of data. No Logger was available.

What I did in order to make it work was to use Trends and have the starting date in the past - 3 months in the past. Trends can retrieve historical data, and since you can have them running on smaller time spans (hourly or daily depending on the number of events and the load on your ESM) they will not be so hard on the ESM resources as a single Query would be.

So:

-> Create a Query for your events; make sure to filter out everything which you do not need

-> Create a Trend running hourly or daily (if you have a large number of events, perhaps it is better to have it on an hourly basis) and have its start date 1 month in the past

-> Monitor the Trend and its progress by using the "Refresh Trend Runs" widget, but do not actually click "OK" because it will start all over again each time. Just choose your time range and click the circular time range refresh button on the right hand side of the time range options.Gradually, you should see as new runs are being added below.

-> Create your subsequent Query/Report on the newly available data from the Trend

I do advice you monitor the Trend while it is gathering the historical data, just to make sure it goes smoothly. This might take minutes or hours, depending on the volume o data. However, considering that it is running on hourly data, it should be able to cope with it.

Good luck,

Stefan

Highlighted
Outstanding Contributor.. mustapha_arakji Outstanding Contributor..
Outstanding Contributor..

Re: run Report on old Data

Nice.

I belive Trends are good candidates for such a case. So using Paul's idea about refining the query and your idea about trends, would generate the results required without effecting system performance.

Mustapha
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.