Why do I encounter errors such as "Link Checking(HTTP: 500 - internal server error)" when performing Link Checking on a URL?
When using the Link Checking function "WebSetOption(WEB_OPT_LINK_CHECK,1)" for checking and validating all Hyperlinks on a Web Page, occasions can arise when this operation fails and you may encounter such error messages as:
Link Checking(HTTP: 500 - internal server error, URL: http://www.server.com/linkname) Link Checking(WebEngine: 28 - Link checking, 1 http(s) links of page http://www.server.com are not working!)
These error messages can occur because the Web API function "WebSetOption(WEB_OPT_LINK_CHECK,1)" issues a HTTP "HEAD" Request to every Link found in the base URL. The most common reason why the Link Checking fails is because the Web Server under test has been configured to reject any HTTP "HEAD" Requests with a HTTP 500 status code, as a security precaution.
Note The role of the HTTP "Head" Header is to issue a request to a server and receive a response header only, it does not expect a HTTP Response body to be returned.