Scanning RESTful WebServices with OAuth Token

I am scanning RESTful WebServices and it is secured by OAuth2 (Spring Security OAuth). These webservices are not accessible from the website, only accessed by other applications as server to server API calls.

Each application requests for the token (POST call) and receives access token, refresh token, expiry duration in response. Every further request sends an unique correlation id, access token as request headers.

I was planning to use web proxy to record the requests, convert to web macro and use workflow driven scan. But since the token expires and correlation id needs to be unique, requests sent from webinspect fail. Is there a way to replace the token , correlation id in the macro prescan or dynamically during the scan?

 

 

  • Provided you have a Login Macro that effectively authenticates you, then you should be able to set these fields as Dynamic during the scan, even if you are using replayed traffic.  To do this, open the Default Scan Settings > HTTP Parsing panel, and add the name of these token field(s) to the upper block for "HTTP Parameters Used for State".  Since I am lazy, I would set this broadly and enable all of the options for where this parameter may be found, e.g. HTTP Query, HTTP POST, et al.

    Declaring the Custom State-Keeping Parameters here informs WebInspect that the value is dynamically arriving from earlier in the scan Requests, and that whatever value the server provided should be used in subsequent Requests, until the server updates them.  So now when you run this scan, the Login Macro will get the new values dynamically, and the HTTP Parsing setting will force WebInspect to carry those new values throughout rather than using the stale values in the recorded Workflow or Start Macro.

    The HTTP Parsing declaration also limits how much WebInspect will fuzz those parameters.  Since you declared it as important for session state, WebInspect will not hammer on the values like it would for any old Mailing Address field (as an example), knowing that that action is likely to foul up the session state.  However, WebInspect will still perform some fuzzing of the parameter(s) declared.

    IF you cannot permit any fuzzing of these parameters, then you must simultaneously and separately declare them elsewhere under the Attack Exclusions scan setting panel.  Adding the same parameter name(s) to the Attack Exclusions prevents WebInspect from performing any auditing of the parameters.  If you investigate that scan settings panel, you will see that it already includes a good listing of known state-keeping items seen in the wild/industry.

     

    You mentioned what sounded like a separate parameter, "unique correlation id".  If the applciation's action is to take the state-keeping parameter's Value and transfer it to be the Value of another parameter used in later Requests, then you will need to get details on how to manage this from our Fortify Support team (support.fortify.com).  There are hidden scan settings that can be used to dynamically reassign a parameter Value to another parameter, if that is indeed your situation.  Usually that occurs when some middle-ware wishes to re-use a Cookie value as a parameter to make a back-office request on behalf of the browsing visitor.  Ultimately, Support can help you review the live Web Proxy captures and the WebInspect scan details (Traffic Monitor!) to help you identify the best configuration to handle this application.

  • Hello, 

    I am facing a similar problem but the solution did not work. Here's some background .... 

    When using WebInspect (17.10), what would be the best way to run REST webservices that use a header value picked from the body of a previous response. The header is an authentication identifier that is set in the response body of a login request.

    Currently I am using burp to capture the requests and response off of Ready API (previously SoapUI), converting them to a webmacro and loading it into webinspect. In the same way, using the login service as a login macro. In this scenario, authentication fails as webinspect is unable to set the token returned in the response to the login request as the header for the services defined in the webmacro. 

    From the pointer provided here, trield using a regex (\btoken\b)  to set the state value in http parser, but that did not help. When looking at the traffic monitor, requests that required a user to be logged in were not authorized. 

    Any help here is much appreciated. Thanks!

  • This is indeed tricky, as your app is shifting the assigned Value for one Parameter to be the Value submitted for a different Parameter later.  This occurs in some middle-ware sites where the Cookie on the front-end is used as a parameter value to make requests on back-end applications within the network.

    There is a hidden scan setting you can add to the raw scan settings file (XML) to define this behavior and specific Parameters to WebInspect.  This configuration or setting is known as "EnableResponseStatePatterns" and "ResponseStatePatterns". Please contact the Fortify Support team (support.fortify.com) for assistance with this.

     

    Here are some basic notes on this feature, but please work with Support to ensure nothing is incorrectly set or omitted for your case.  A Web Proxy or BURP capture of the offending behavior would be excellent to help them understand how it operates in your situation.

    • CONFIGURATION:
    1. The first thing is to set the EnableResponseStatePatterns to true (default is false).
    2. Then add a ReponseStateElement for each variable being tracked.
    3. Each response state element (under ResponseStatePatterns) represents a single state value. Search regexes allow us to determine where we collect state from in HTTP Responses. The Named Group is important as it tells us which part of the match in the regex is the actual state value. Replace regexes tell WebInspect where to apply that state on HTTP Requests. Again the Named Group (name must match the name of the response state element, in this case “Oracle”) is required to tell us where in the match the actual state value goes.
    <EnableResponseStatePatterns>true</EnableResponseStatePatterns> <ResponseStatePatterns> <ResponseStateElement> <name>Oracle</name> <SearchRegexes> <string>Location:.*f\?p=[0-9] :[A-Za-z] :(?<Oracle>[0-9] ):::</string> </SearchRegexes> <ReplaceRegexes> <string>&p_instance=(?<Oracle>[0-9] )&</string> </ReplaceRegexes> </ResponseStateElement> </ResponseStatePatterns> 

     

    • You will find a self-closed element that looks like <ResponseStatePatterns />
    • Expand this element and fill it in as needed to look as follows:
              <EnableResponseStatePatterns>true</EnableResponseStatePatterns>  <ResponseStatePatterns>  <ResponseStateElement>  <name>CaptureGroupName</name>  <ReplaceRegexes>  <string>RegexWithCaptureGroupToApplyStateToRequest</string>  </ReplaceRegexes>  <SearchRegexes>  <string>RegexWithCaptureGroupToCollectStateFromResponse</string>  </SearchRegexes>  </ResponseStateElement>  </ResponseStatePatterns> 
    • The SearchRegexes are applied on the entire HTTP Request and the ReplaceRegexes are applied on the entire response. The capture group would represent the exact token that needs to be transferred from the HTTP Response to the following HTTP Request. So, there is no need to worry about a parameter name in this technique. All that matters is the exact value that represents the state.
    • EXAMPLE:
      • Given that the HTTP Response Body returns this: ... {"Token":"2bfcf7e0-5b31-46e8-aabf-007e26a36177","Overlayreqd":1,"Disabled":0,"Downloadpriv ...
      • And that the HTTP Request is seeking this header: Authorization-Token: 2bfcf7e0-5b31-46e8-aabf-007e26a36177
      • While the important Value is: 2bfcf7e0-5b31-46e8-aabf-007e26a36177
      • Then this is mostly likely how the scan settings (regex) can be used:
    <EnableResponseStatePatterns>true</EnableResponseStatePatterns> <ResponseStatePatterns> <ResponseStateElement> <name>Token</name> <ReplaceRegexes> <string>Authorization-Token:\s(?'Token'[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12})</string> </ReplaceRegexes> <SearchRegexes> <string"Token":"(?'Token'[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12})"</string> </SearchRegexes> </ResponseStateElement> </ResponseStatePatterns>