Contributor.. Aneesh Salimkumar Contributor..
Contributor..
765 views

ArcSight ESM API for Getting the Query Viewer Data

Jump to solution

Hi, 

We are trying to establish some visualization (using a customized portal) utilizing the data inside ArcSight. We are looking to integrate using REST API calls to ArcSight ESM. We can successfully obtain event information using the API now.  We are trying to do the same with QueryViewer. 

However, I am not getting any luck with the response. 

I already went through this discussion and I am trying to test the QueryViewer service by GET URL mentioned in this discussion

https://community.softwaregrp.com/t5/ArcSight-User-Discussions/Understanding-ESM-API-documentation/td-p/1657890

And I am trying this URL with and obtained authtoken and the resource ID for a queryviewer. I am getting a 404 response. I tried with different resource IDs but the result is same. I am very new to the ESM API, Can any body guide me fix this problem

Hopefule get some help from @Marius2 

Best Regards,

Aneesh Salimkumar

 

 

 

0 Likes
1 Solution

Accepted Solutions
Highlighted
Micro Focus Expert
Micro Focus Expert

Re: ArcSight ESM API for Getting the Query Viewer Data

Jump to solution

Hello @Aneesh Salimkumar 

The issue mostly is that GET requests has some issues with encoding of the URL, while + usually remains a plus for example, ESM recognize it as the ASCII representation of space instead.

I use POST requests for all my calls whenever I can to ensure no further issues, so I would recommend doing the same!

Here is an earlier example that I posted:

 

import requests
import json

def get_query_viewer(authtoken, qvsresourceid):
    headers = {'accept': 'application/json'}
    url = 'https://ESM:8443/www/manager-service/rest/QueryViewerService/getMatrixData'
    payload = {
        "qvs.getMatrixData": {
            "qvs.authToken": authtoken,
            "qvs.resourceId": qvsresourceid,
        }
    }
    response = requests.post(url, json=payload, headers=headers, verify=False)
    return response
-----------------------------------------------------------------------------------------
All topics and replies made is based on my personal opinion, viewpoint and experience, it does not represent the viewpoints of MicroFocus.
All replies is based on best effort, and can not be taken as official support replies.
//Marius
3 Replies
Highlighted
Micro Focus Expert
Micro Focus Expert

Re: ArcSight ESM API for Getting the Query Viewer Data

Jump to solution

Hello @Aneesh Salimkumar 

The issue mostly is that GET requests has some issues with encoding of the URL, while + usually remains a plus for example, ESM recognize it as the ASCII representation of space instead.

I use POST requests for all my calls whenever I can to ensure no further issues, so I would recommend doing the same!

Here is an earlier example that I posted:

 

import requests
import json

def get_query_viewer(authtoken, qvsresourceid):
    headers = {'accept': 'application/json'}
    url = 'https://ESM:8443/www/manager-service/rest/QueryViewerService/getMatrixData'
    payload = {
        "qvs.getMatrixData": {
            "qvs.authToken": authtoken,
            "qvs.resourceId": qvsresourceid,
        }
    }
    response = requests.post(url, json=payload, headers=headers, verify=False)
    return response
-----------------------------------------------------------------------------------------
All topics and replies made is based on my personal opinion, viewpoint and experience, it does not represent the viewpoints of MicroFocus.
All replies is based on best effort, and can not be taken as official support replies.
//Marius
Contributor.. Aneesh Salimkumar Contributor..
Contributor..

Re: ArcSight ESM API for Getting the Query Viewer Data

Jump to solution

Thank you Marius for the prompt response. Much appreciated. I have made the script to work now by replicating get_event function with get_queryviewer (I am very new to Python). Then I saw your response which is awesome. 

I am able to get the results into a file using below code.

filename='test.json'
with open(filename,'w') as fp:
json.dump(event_details, fp)

 

Now I am trying to do the proof of concept to have visualization platforms like Dundas or Power BI and use the data obtained through this API visualize. I am not sure if this is something commercially offered from Microfocus. 

JSON format looks too complex (for the right tabular format) to be parsed directly with Power BI. Is there a way to somehow simplify the output format.

Best Regards,

Aneesh

 

 

 

0 Likes
Micro Focus Expert
Micro Focus Expert

Re: ArcSight ESM API for Getting the Query Viewer Data

Jump to solution

Hey @Aneesh Salimkumar !

Actually the nice part about python and retrieving data from ArcSight, meaning you are in full control of the format yourself makes it soo much easier.

This way you need to look at what format the third party system expects, and just reformat the data in python.

Iwe built large scale visualizations on top of ArcSight before, utilizing mostly the queryviewer api (as it allows you to format the output of your data itself, because what you see in the queryviewer is what you see in the API (and then use SecurityEvent API for drilldowns on specific events).

If you can give an example JSON or CSV of the expected output + a copy of the format from your queryviewer then i can point you in the right direction.

Worst case I am also part of professional services at Micro Focus, so PM me if there is any interest to aquire that type of service, and we can make the baseline for you.

-----------------------------------------------------------------------------------------
All topics and replies made is based on my personal opinion, viewpoint and experience, it does not represent the viewpoints of MicroFocus.
All replies is based on best effort, and can not be taken as official support replies.
//Marius
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.