Having problems with your account or logging in?
A lot of changes are happening in the community right now. Some may affect you. READ MORE HERE
gp1988 Trusted Contributor.
Trusted Contributor.
5876 views

Cloud Scan / Very Powered Machine Performance Comparison

Jump to solution

Hello,

I would like to know your opinion about the following problem.

I have to setup a Fortify installation on a system that manages many projects: there is a medium level of parallelism (max 8 parallel scans to do ) and 80% of small projects (10000 LOC), while 25% of great projects (>=30000 LOCs), about 5% very very large project (500k LOCs). Main languages are Java and JS.

A Jenkins machine have to start scanners and the most simple configuration is to leave a single machine that build, compile, translate and scan by upgrading the hardware of Jenkins Machine in order to improve performance. 

The problem is that there is a sort of performance requirement: the maximum time of a scan should be 20 minutes. There is no problem for low projects, but for very very large projects (500K) I think that SCA will never be able to do a full scan  in 20 minutes even if I ask 256 GB of rams and 128 Cores (do you think that it would be possible?) ...

Minimal hardware required from my team is 8 cores and 32 dedicated GIGA for the Jenkins machine. 

According to your experience do you think that it will be possible to scan the large projects in required time with this configuration or with CloudScan? 

And, in your opinion: 

  • In order to have the best performance for a large project is it better to have 1 dedicated sca machine with 32 GB of RAM and 8/16 cores or a configuration with CloudScan of 4 machines with 4 cores and 8 GB?
  • Does CloudScan distribute a single scan between sensors (for example each sensor uses an analyzer, then there is a merged result into an fpr) or choose a single sensor for each scan (i.e. if you have 4 projects to scan, each sensor does a full scan for a single project)?

Thanks very much in advance! 

Best Regards

0 Likes
1 Solution

Accepted Solutions
Highlighted
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Re: Cloud Scan / Very Powered Machine Performance Comparison

Jump to solution

Good day  GP1988

The Cloud scan upload command is designed to for1 MBS Session for upstream import and scanning on 1 Scanner.

If your local translate SCA command causes a number of projects (a,b,c,d, -N) to create .nst  for 1 buildid (buildid = folder). 

The .nst files in the buildid folder are included in the MBS file and sent (imported) to 1 scanner.   The MBS work is never split between accross scanners.  The output is 1 FPR for all Project .nst  files created/included in the transalate process are scanned together.

Best, Paul

 

Tags (1)
4 Replies
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Re: Cloud Scan / Very Powered Machine Performance Comparison

Jump to solution

Good day GP1988

Scan time is a function of number of elements (code/hardware) and can be finetuned.  Scan time is not directly releated to the number of programs or  lines of code, rather ELOC and complexity are more important.

Items that can affect scan time:

CPU/CORES/MEM,  Existing CPU program load,  RulePack version SCA Version, Scan Depth (see SCA quide for 'quick scan'), Type of Code, Code Complexity, High order analysis (HOA)  etc   

See 17.20 Performance Guide attached hereto.

FYI: 17.20 is configured for mutlithreading by default (see fortify-sca.properties)

    com.fortify.sca.MultithreadedAnalysis = true

    com.fortify.sca.Phase0HigherOrder.Languages = python,ruby,swift

You should establish a 'time' base line for each of your scans and explore Performance options.

Include -debug -logfile <translate.log> for the SCATranslate command and -debug -logfile <Scan.log>  for the SCA Scan command.   Review the logs to learn what SCA has Translated and where SCA is spending SCAN time. (note: be sure to resolve Translation issue (missing Classes, etc) before executing a SCAN)

Example: 3-5 hours is not an abnormal time for a scan of 4,000 Java Programs.

Note: SCA is configured to balance best practices for   Time   Vs.   Depth   of scan.

WIth regard to Projects.

A.) If a Master project is composed of 4 subproject they should be scanned 'together' to allow the Dataflow Analyzer to see the full scope of the program.  The finished FPR represents the 'entire' master project.

B.) By default when merging FPR Fortify assumes each FPR is the 'Same' project, just successive interations.

Program A-V1;  Program A-V2,   Program A-V3       If you are not merging with previous scan each scan represents the a unique scan for the code at that moment in time.

If you are loading SSC with each FPR over time SSC will merge and will determine VULNS tthat are NEW, Fixed(removed), and Existing.

(There are special directives to merge unrelated project FPRs for reporting needs)

Cloud scan does 'not distribute' a Master project composed of 4 subproject (a, b, c, d) to multiple sensors and then merge FPRs a,b,c,d.   The scanner will scan the entire project (a,b,c,d) resulting in one complete FPR.

Kind regards

Paul Caliban

Fortify Customer Support

0 Likes
gp1988 Trusted Contributor.
Trusted Contributor.

Re: Cloud Scan / Very Powered Machine Performance Comparison

Jump to solution

Hi Caliban,

thanks for the answer! 

About the CloudScan question, maybe I was not able to explain my question. 

The question was not about how to separate a project X composed by a,b,c,d subprojects in 4 different sensors, but was to know if a project X is analyzed by just one sensor present in Cloud Scan, or it is sent on all 4 different sensors, and one sensor scans for control-flow analys, another sensor for configuration analysis, another for static analysis, etc. with a final FPR that joins all the analysis.

Otherwise, the Cloud Scan works in a different way: the project X is sent on a single sensor that will scan with all analyzers, and another project Y is analyzed by another sensor. 

Thanks very much ! 

 

 

 

 

 

 

 

0 Likes
Highlighted
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Re: Cloud Scan / Very Powered Machine Performance Comparison

Jump to solution

Good day  GP1988

The Cloud scan upload command is designed to for1 MBS Session for upstream import and scanning on 1 Scanner.

If your local translate SCA command causes a number of projects (a,b,c,d, -N) to create .nst  for 1 buildid (buildid = folder). 

The .nst files in the buildid folder are included in the MBS file and sent (imported) to 1 scanner.   The MBS work is never split between accross scanners.  The output is 1 FPR for all Project .nst  files created/included in the transalate process are scanned together.

Best, Paul

 

Tags (1)
gp1988 Trusted Contributor.
Trusted Contributor.

Re: Cloud Scan / Very Powered Machine Performance Comparison

Jump to solution

Thansks very much, it's exactly the answer to my question ...

 

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.