Highlighted
Sethu_BNYMT Regular Contributor.
Regular Contributor.
4155 views

WebInspect Crawl continues in Audit Mode

HI,

Im using WebInspect 18.10. I initiated a scan for our webapplication with "Crawl Only" mode. I used workflow macros to crawl the application. The crawl completed after sometime crawling many additional pages and scan entered the completed state. When I clicked on Audit icon, the WebInspect continued to crawl even during the Audit phase. Since ours is a big application, the scans appears to be never ending.

My question is 2 parts (1) Is it possible to restrict Webinspect to crawl only the pages covered in Workflow macro instead of crawling additional pages? (2) Is it possible to restrict/instruct Webinspect not to crawl during Audit phase?

NOTE: List driven scan is not suitable for our target application.

Recommendations from the community is much appreciated.

0 Likes
3 Replies
Micro Focus Expert
Micro Focus Expert

Re: WebInspect Crawl continues in Audit Mode

Sethu_BNYMT;

Workflow-Driven scanning works best with the Audit-Only method.  The assumption is that if you went to the trouble to pre-record specific sessions and pages, then that is all you are interested in testing.

When you coupled the Workflow-driven option with Crawl-Only, you effectively force-fed the Crawler engine a series of known sessions, and then let it go from there until it completed the Discovery of the entire site, just as if you provided it only a single Starting URL.  However, a risky item here is that with a Workflow-driven scan, WebInspect discounts Disallowed Hosts, instead assuming that you really do wish to test all the Hosts (Allowed Hosts) captured in the Workflow.  Adding that logic to a Crawl-Only, you can see how the Crawl might expand in scope beyond your desired target.  You might be able to prevent that by manually deleting offending sessions from your Workflow Macros, or by adding select Session Exclusions for ("Host"+"Matches"+"enter_undesired_host_name"), but those excluded sessions might negatively affect the playback of your selected Workflow Macros.

Next, when Auditing, if any new areas are exposed through the probes, then those are handed back tot he Crawler to attempt to run further Discovery on those links, and then those are Audited as well.  this is part of our Recursive Crawl and Audit features.  This may be where your Audit began discovering additional areas for the site(s).  The Recursion setting is found on the General scan settings panel, but decrementing it from 2 to 1 or 0 may still not prevent these additional Discovery actions.  An additional setting to consider could be on the Method scan settings panel, for Crawl and Audit Details.  Again, this may still not eliminate all forceful browsing and additional Discoveries made during the Audit phase.


-- Habeas Data
Micro Focus Fortify Customers-Only Forums – https://community.softwaregrp.com/t5/Fortify/ct-p/fortify
sethumady
New Member.

Re: WebInspect Crawl continues in Audit Mode

Hans,

I followed your first recommendation of using the workflow driven macro in audit only mode. And it solved my original problem. Thanks a lot for that.

But I got into a tricky situation where the site tree generated with the same set of workflow macros differed for each and every scan. Do you have any suggestions/reasons for this kind of behavior?

0 Likes
Micro Focus Expert
Micro Focus Expert

Re: WebInspect Crawl continues in Audit Mode

Dynamic scans are not perfectly ordered, and so there can be extraneous sessions requested or recorded form scan-to-scan.  this can be based on network issues, which session was added to the request queue in which order, differences in the session details (page plus parameters), et al.  If you are seeing dramatic differences, please take the scans to Fortify Support (softwaresupport.softwaregrp.com) for a closer review.

 


-- Habeas Data
Micro Focus Fortify Customers-Only Forums – https://community.softwaregrp.com/t5/Fortify/ct-p/fortify
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.