Absent Member.
Absent Member.
7433 views

webinspect how to scan both http & https


I need to schedule a scan that performs both http & https crawling and audit of a target site

If I insert a FQDN (like www.mydomain.it) Webinspect scans only port 80 and I see in disallowed hosts  HTTPS port that is out of scope of my scan

If I use HTTPS://... only port 443 is scanned

how can be scheduled a scan in order to automatically perform HTTP & HTTPS crawling & auditing?

thanks

SA

Labels (1)
0 Likes
5 Replies
Micro Focus Expert
Micro Focus Expert

You need to add it under the Allowed Hosts scan seeing. This can be done manually within the Default Scan Settings prior to the scan wizard being opened, or in the Settings options found within the scan wizard itself (the "Current Scan Settings"). Both the Guided and the Basic Scan Wizards should highlight this second Host during the Profiler run, if there is a connection made early in the browsing of the Starting URL. If the connection never occurs until after authentication or beyond, it is possible the Profiler (50 HTTP Requests by default) would not see it for you. If present, your Login Macro should expose this connection to the Profiler. Additionally, you could force a connection with a single page Workflow Macro (likely want to change the scan mode from it expected Audit Only back to Crawl and Audit), or by manually browsing to it during the Scan Optimization phase of the Guided Scan Wizard.

Sent from my Verizon 4G LTE smartphone


-- Habeas Data
Micro Focus Fortify Customers-Only Forums – https://community.softwaregrp.com/t5/Fortify/ct-p/fortify
0 Likes
Absent Member.
Absent Member.

Thanks for the answer,

but unfortunately allowed hosts is not a valid option, because the name is the same

I have verified that if I insert in the allowed hosts the URL https://... with the same FQDN, in the wizard during profiling, then HTTPS:// is cut off and remains only the FQDN... so no HTTPS is scanned

While if I open the session AFTER the scheduled scan I can add the excluded host that is reported correctly with HTTPS in the excluded hosts

My big problem is that I have to schedule a lot of different hosts to scan (anonymously), and I cannot edit manually for modification in the current scan setting for each hosts...

In default scan settings I have not found any option to allow http&https crawling/auditing

Thanks

SA

0 Likes
Micro Focus Expert
Micro Focus Expert

On a review of the Sample Scan and the Help file, I realized that you cannot include the protocol precursor, but you can specify the port number, or simply the root host name as a "wildcard".  It appears that this Allowed Hosts settings operates like a Regular Expression and not purely text matching.  The Sample Scan includes both hosts/ports http://zero.webappsecurity.com (port 80) and https://zero.webappsecurity.com (port 443), and the Allowed Hosts entry there shows "zero.webappsecurity.com:443".  I have attached a screen shot of that.

If you are using automation such as the WebInspect API, there are Override options so that when you specify a saved scan settings file, or the Defaults, you can overlay select settings in the command.  This would permit you to script an Override to specify the host name and/or its ports for that Allowed Hosts setting.

+++++++++++++++++++++++++

From the Help file:

Scan Settings: Allowed Hosts

To access this feature, click the Edit menu and select Default Scan Settings or Current Scan Settings. Then, in the Scan Settings category, select Allowed Hosts.

Using the Allowed Host Setting

Use the Allowed Host setting to add domains to be crawled and audited. If your Web presence uses multiple domains, add those domains here. For example, if you were scanning "WIexample.com," you would need to add "WIexample2.com" and "WIexample3.com" here if those domains were part of your Web presence and you wanted to include them in the crawl and audit.

You can also use this feature to scan any domain whose name contains the text you specify. For example, suppose you specify www.myco.com as the scan target and you enter "myco" as an allowed host. As Fortify WebInspect scans the target site, if it encounters a link to any URL containing "myco," it will pursue that link and scan that site's server, repeating the process until all linked sites are scanned. For this hypothetical example, Fortify WebInspect would scan the following domains:

  • www.myco.com:80
  • contact.myco.com:80
  • www1.myco.com
  • ethics.myco.com:80
  • contact.myco.com:443
  • wow.myco.com:80
  • mycocorp.com:80
  • www.interconnection.myco.com:80

Adding Allowed Domains

To add allowed domains:

  1. Click Add.
  2. On the Specify Allowed Host window, enter a URL (or a regular expression representing a URL) and click OK.

    Note: When specifying the URL, do not include the protocol designator (such as http:// or https://).

Editing or Removing Domains

To edit or remove an allowed domain:

  1. Select a domain from the Allowed Hosts list.
  2. Click Edit or Remove.

-- Habeas Data
Micro Focus Fortify Customers-Only Forums – https://community.softwaregrp.com/t5/Fortify/ct-p/fortify
0 Likes
Absent Member.
Absent Member.

Thanks hans

I have tested your solution...

although it appears correctly in the allowed hosts, the final result of the scan is not what I want

I have also inserted specific port :80 and :443 in order to force the analysis on those ports but result is not the same

If you perform analysis of a test site like

http://zero.webappsecurity.com and

https://zero.webappsecurity.com

you receive different issues for each scan

what I need is the UNION of the issues with a single scan, and the only solution I found (tat is very expensive or impossible on big sites) is to perform an enterprise scan of both URLS

0 Likes
Micro Focus Expert
Micro Focus Expert

You may want to contact Fortify Support (support.fortify.com) for a personal review of your scan settings and the targets you wish to be in scope.

An Allowed Host entry of "www.mydomain.it" should permit scanning both ports, or any permutations for that matter, of that host name.  Or simply having "www.mydomain.it:443" as the Allowed Host entry should include that host when scanning with "http://www.mydomain.it " as the Starting URL.

Regardless of how you enter the Allowed Hosts entry, one hard requirement is that there is at least a single link found on the targeted host (per the Starting URL field) that directs the Crawler to that secondary host.  If there are truly no cross-links between these two apps/ports, then you must force its "discovery" by adding at least a single HTTP Request to that secondary host with either a Workflow Macro, a Start Macro (Authentication scan settings panel), or by manually browsing to it during the Scan Optimization phase found within the Guided Scan Wizard.


-- Habeas Data
Micro Focus Fortify Customers-Only Forums – https://community.softwaregrp.com/t5/Fortify/ct-p/fortify
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.