joe.yeager Absent Member.
Absent Member.

Check Your Timeouts

At a customer site the other day I saw that they were having a problem with timeouts. Timeouts are something that many people will overlook, but they are very important because they can usually lead to inaccurate scan results.

Typical reasons for timeouts to occur: 

  1. You may be experiencing serious network problems
  2. There may be an IPS or web app firewall between you and your target machine
  3. The web server or web application may be down
  4. Your scan settings may be too rough on the application

WebInspect will retry the attack as many times as your settings specify but it will sometimes falsely flag on things like buffer overflows, not find pages, and worse yet, not find critical vulnerabilities.

Troubleshooting Timeouts

When an attack times out it is usually the result of the web app, the server, or the network closing the connection. To better determine which is the result, go to the scan log tab and look at the requests that are failing (the ones in red). To better examine the requests, double-click on the red text and scroll to the right.  As an error message you should see something similar to "Request timed out", "Connection forcibly closed", or "Connection closed by remote host".

If sequential requests are failing... may have lost your connection to the server, or the server (or application) may have gone down. Verify using your web browser that the app is indeed working and the server is up as well. On one customer site, WebInspect’s crawler found the administration section of the application’s framework and took the site down. If something like this is the case, make sure to exclude that section by un-selecting that area in your scan tree, or adding some values to "Excluded URLs" in your default scan settings. If the application is indeed up, change your timeout settings to the ones at the bottom of the page. may have hit a problem page in the application.  If you are currently on one of the parameter manipulation engines (i.e. Query Injection, Postdata Injection, etc) and all-of-a-sudden every request is failing, check the text in red.  If it is the same filename every time, just different parameter attacks, you definitely hit a problem page.  Some applications have pages that do not respond well to garbage stuck in their parameters.  Rather than give an error message back to the user, they simply dangle the connection and WebInspect slowly times out.  These problems should be reported to the developers and the problem page noted for further evaluation.  There could actually be a vulnerability here, but there is definitely a problem with quality of service. 

If random requests are failing... may have an IPS (intrusion prevention system) or a webapp firewall between you and your target machine. If attacks fail that are signature attacks like "boot.ini" and "etc/passwd" but other requests seem to be fine, this is most definitely the case. Some security professionals do not know whether they have one of these devicess in between them and the application, and only by seeing WebInspect timeouts and then asking their networking teams do they find out. While scanning with one of these devices turned on is a good test of the device, you will want to turn it off (or ask the networking guys for a bypass) for a comprehensive vulnerability test of the web application. Just remember, it is better for you to find it through security scanning then a hacker down the road. may have hit the limit of the application. See the section directly below.

If you are getting an extraordinary amount of timeouts... may have an application that cannot handle the load of a web scan, or the amount of concurrent requests that we are making at a time. Sometimes applications closely watch a user’s session state, and when we make several requests at a time using the same session state, the application hangs. To prevent this, make the settings changes at the bottom of the page. If this does not help, you should consider speaking with the developers about the problem. With these slower settings WebInspect should not put a larger load then about 3 concurrent users of the application. You might also ask about putting the application on a better server.

Recommended WebInspect Settings

These settings are designed to slow a scan down as a result of timeouts. To change these in WebInspect, go into Tools -> Default Settings.

As is default, make sure the “Scan Properties” section on the left-hand side of the “General” tab is selected

Request Timeout: 60 (secs)
     Reason: if you increase the time until a request times out, it will allow more time for a slower application (or server) to respond.

Retry Count: 5
     Reason: if you increase the retry count, when a request fails WebInspect will retry it again and again, making sure that the attack takes place. Note: the total amount of requests is calculated like so: 1 (original) + 5 (retry count) = 6 total requests

Thread Count: 1
     Reason: if you lower the amount of threads the scan will take longer, but WebInspect will be much gentler on the application (and server) and you should fix most of your timeout problems.

Click on “Advanced” on the left-hand side of the “General” tab

500 Total Timeouts
     Reason: if you are having problems with the scan pausing after you reach a bunch of timeouts, increase this value. If after 500 total timeouts and the scan still pauses, you are definitely doing an inaccurate scan and you should consult your development and networking teams.

50 Consecutive Timeouts
      Reason: if you see 50 timeouts in-a-row you have probably lost your connection to the server or the application has gone down, and 20 requests (the default) can be the result of an IPS or web app firewall.

Labels (1)
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.