Our vBulletin migration is complete.
Welcome vBulletin users! All content and user information from the Micro Focus Forums (vBulletin) site has been migrated to this site. READ MORE.
Highlighted
Respected Contributor.. dgarozzo Respected Contributor..
Respected Contributor..
1845 views

SSC Application Version Git Jenkins Automation Workflow FPR Merging

My current setup:

Git is organized with a develop branch as the main branch, and Feature branches as child branches off of develop. Feature branches are merged into the develop branch with a pull request.

Jenkins build jobs invoke a Fortify scan of Git branches. Fortify SSC utilizes the branch names as the version names, so each feature branch, as well as the develop branch, is scanned as a separate application version.

When a feature branch is first created, it is initialized with the FPR file from the develop branch to keep any existing auditing that has been done.

In an effort to reduce duplicate auditing work, does it make sense to upload the Feature branch's FPR file to the develop branch upon a successful Pull Request? That is, I'm thinking that I would merge the code into the develop branch, upload the Feature branch's FPR file to the develop branch, then kick off a new code scan of the develop branch after the code has been merged. Would this allow me to retain any auditing that had been done on the Feature branch? Would this mess anything up (metrics)?

What about when a feature branch pulls code from the develop branch to get the latest updates from develop? Should I upload the FPR from develop to the Feature Branch and then scan the newly merged code?

If I had multiple feature branches going at the same time, and they all submitted pull requests without first updating from the develop branch, and I uploaded the FPR file from each feature branch to the develop branch and re-scanned each time, would that work? Would that retain all of the audit information from all of the Feature branches as well as the Develop branch?

Does this flow make sense?

Labels (3)
0 Likes
5 Replies
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Re: SSC Application Version Git Jenkins Automation Workflow FPR Merging

Good day Dgarozzo

Not sure we completely understand your SSC confg or goal for Vuln Reporting and need for dataflow between DEV Mainline and each Feature Branch

It is not clear what you are scanning and if you are doing a -clean step in your flows or retaining .nst translations.

It is not clear if you report on just Feature Branches  or if  Features are always scanned as part of the larger MainDEV trunk.


So this is statement is not clear:  It seems you want to COMBINE  vs MERGE (see details below)
<quote>When a feature branch is first created, it is initialized with the FPR file from the develop branch</quote>

Do you scan:
The entire project with all features 'checked' in?
or
Just one Feature Branch by itself? Then later hope to push those Feature Branch vulns into the exiting MainProject replace previous Feature Branch vulns?

The retention of 'previous' audit information is dependent on the INSTANCEID value. (a completely different question)

Attempting to make SSC look like your GIT structure by not meet your SSC reporting goals.

It might be best to explain the following first.

SSC does not 'combine' FPRs; SSC 'merges' FPR Data
THIS is KEY!


Fortify products assume your scans are for the 'same' application 'not' a group of different applications/features that may or may not be related.


Meaning: 'combine' N-Number of Project (FPRs) (FPR Project1 + FPR Different Project2, etc)
Vs
'Merging' A series of FPRs for 'A' Single Project (Project A - Version FPR1A +Version FPR2A + Version FPR3A, etc)

All the Fortify Programs operate the same. The all 'merge' data  vs  'combine'
'combine' is additive
'merge' is 'not' additive.

AWB Merge
FPR Utility Merge (Note: The LOC/ ELOC and FILE count will be pulled from Last FPR)
SSC Merge

===========================================
IF COMBINE is needed use the "APPEND" option.

sourceanalyzer -b “Translate A” -scan -f Combine.fpr <<< create1st FPR with A Vulns
sourceanalyzer -b “Translate B” -scan -f Combine.fpr -append <<<Adds B Vulns to A Vulns
sourceanalyzer -b “Translate C" -scan -f Combine.fpr -append <<< ADDs C Vulns to B&A Vulns

The LOC/ ELOC and FILE count will be cumulative

0 Likes
Respected Contributor.. dgarozzo Respected Contributor..
Respected Contributor..

Re: SSC Application Version Git Jenkins Automation Workflow FPR Merging

The develop branch is what would ultimately get compiled and pushed to production. As developers make their code changes, they make a feature branch off of the develop branch. When those changes are done, they push their changes back into the develop branch.

We want to scan the code at the develop branch level so we can prevent issues from going in to production.

We want to scan the code at the feature branch levels so developers can see and track their issues before pushing their code to the develop branch.

We have set things up in Fortify such that an application's branch name in git is included in the application's version name in Fortify.

When a feature branch is created, I am creating a new application version in Fortify SSC and am initializing it with the FPR file from the develop version. This preserves any auditing and comments that have been done in the develop version so they can be seen in the feature version.

Audits and commenting are happening in the feature versions. When work on the feature version is done and the code is to be merged in to the develop version, I want to preserve the audits and comments that were added while in the feature version.

Let's say, for example, a developer made code changes in the feature version, and Fortify raised a flag for a vulnerability that we agree to suppress. I can suppress it in the feature version, and the developer can continue doing his development. When a scan is run on the feature version, the report will now come back clean. But after I merge the code into the develop version, that item that I had suppressed in the feature version would no longer be suppressed.

I would like to be able to avoid having to re-suppress an item that we already agreed would be suppressed. (and I would like to keep any comments that were added to the issues in the feature version).

All scans are -clean. No .nst files are retained. All versions are scanned independently.

I'm pretty sure that I don't want to combine data - I want to merge it.

I'm thinking that I should be able to take the latest FPR file from the feature version, upload it to the develop version in SSC, where I believe it would be merged in with the develop version data. Then when the feature version code is pulled into the develop version code, and the merged code is Fortify scanned and uploaded back to SSC into the develop version, it should include the combined audit information from both the develop version and the feature version. Does that sound like it would do what I want?

0 Likes
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Re: SSC Application Version Git Jenkins Automation Workflow FPR Merging

Thanks for the update:

So MERGE is your goal.

<quote>
We want to scan the code at the develop branch [Trunk] level so we can prevent issues from going in to production.
</quote>

Your scan at the "develop branch level" must always be 'merged' to a like "develop branch level"

Meaning all the "feature branches" are checked in so SCA can do a full dataflow analysis - Devmain+Feature-A+Feature-B

Apples Compared to Apples

Yes, Coping/making a new version allows you to discretely track changes over time and preserve all VULNs seen with each Development Branch full Scan

Allow for history reporting:
Project X
Release 1 - Devmain+Feature-A+Feature-B <<
Release 2 - Devmain+Feature-A+Feature-B << new version(seeded with Release 1 upon create)

Some teams only have one version and do not need to do report on Release 1; Only the Last uploaded or single Release

======

As for the Branches. Feature-A; Feature-B; Feature 'N'

You could treat each as standalone project to help the Developer team find 'Feature' only issues.
SCA will not be able to perform a 'full' dataflow between the "features" and the other exiting Develop [trunk] branch code.

Notwithstanding, when a 'feature' is updated in the Develop [trunk] branch and a full 'new' scan is done, other 'feature' VULNs maybe discovered.
Dataflow discoveries, etc
Different Source-Sync paths


<quote>
I'm thinking that I should be able to take the latest FPR file from the "feature version", upload it to the develop [trunk] version in SSC,
where I believe it would be merged in with the develop version data.
</quote>

***This will 'not' work. ***

Given the understanding the develop [trunk] version is =(DevMain+Feature-A+Feature-B)

You cannot create a Release X = .....>> Which is: DevMain+Feature-A+Feature-B and hope to take a standalone Feature-B(new) scan and merge into an exiting DevMain+Feature-A+Feature-B
resulting in a Release Y: DevMain+Feature-A+Feature-B(new)

SCA will "not" dump the original Feature-B and replace with Feature-B(new)

Even more important is when SSC Loads only the Feature-B(new) into an existing project defined as: DevMain+Feature-A+Feature-B
SSC will think your DEVMain+Feature-A Data is gone (aka fixed), because it only sees the Feature-B(new) vulns.

What you are trying to accomplish is not native to SCA.
SCA compares 'like' scans.
I.E V1 = DevMain+Feature-A+Feature-B **compared** to next V2= DevMain+Feature-A+Feature-B **compared** to V3=DevMain+Feature-A+Feature-B

When needed in Rare cases:
You can obtain/control a full new scan with full dataflow analysis, but this will "not" work with finished FPR's;
rather you would need to retain the original (no -clean) DevMain .nst + Feature-A .nst + Feature-B(new).nst files and put into one buildID folder and run a new FULL SCA scan

You would only need to translate Feature-B(new) .nst and combine with original (no -clean) DevMain .nst + Feature-A .nst 

If you control the .nst files you can mix and match (copy) into a new buildID folder. Create empty folder: copy DevMain .NST files; copy Feature-A.NST files and Copy Feature-B(new).NST files

Resulting in an FPR with DevMain+Feature-A+Feature-B(new); which can be loaded to an older SSC DevMain+Feature-A+Feature-B version.


HOWEVER, your notes and suppressions are at 'risk' because SCA, depending the code changes, may not generate the 'same' instanceID values seen in the original DevMain+Feature-A+Feature-B scan.
In all cases you must have the entire application's parts.

 

 

0 Likes
Respected Contributor.. dgarozzo Respected Contributor..
Respected Contributor..

Re: SSC Application Version Git Jenkins Automation Workflow FPR Merging

Let me try to explain this another way.

I've got an application version, A, that I have in SSC.

I create application version B as a clone of A. (doing a build and scan of B would result in the exact same results as a build of A)

I make changes to B, scan and upload to SSC. Results are audited (collaborative audit), notes are made. I make more changes to B, scan and upload to SSC. Results are audited (collaborative audit), notes are made. I make more changes to B, scan, and upload to SSC. Results are audited (collaborative audit), notes are made. Each scan is a clean scan, and the FPR is uploaded to SSC. All auditing is done via either within SSC or Audit Workbench with Collaborative Audit. All audits, notes, suppressions, etc, are retained throughout the process while version B is being worked on.

I now want to take the final state of the changes made in version B and merge them into version A. So, it's as if I checked out version A, made all of the changes that I needed to make (the final state of version B), and checked the changes in. Then I scan version A and upload the results to SSC.

The problem that I have is that if I had audited something in version B that I suppressed, it won't be suppressed in version A. Or if comments were added to a finding in version B, those comments are not retained in version A.

I should be able to keep the audits, notes, suppressions, etc from version B and merge them into version A, so I don't have to redo all of that work.

Once the merge is complete, I can delete version B from SSC, because I don't care about it. I really only care about version A from a reporting perspective.

 

0 Likes
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Re: SSC Application Version Git Jenkins Automation Workflow FPR Merging


Thank you for the updated information
Much clearer : - )

<quote>

The problem that I have is that if I had audited something in version B that I suppressed, it won't be suppressed in version A. Or if comments were added to a finding in version B, those comments are not retained in version A.

</quote>

Suppressions and Comments are attached to the VULN by "InstanceID" value; not the VULN in a Program

What is more than likely happening to you is the following:

Lets assume you have a program with an XSS VULN in VERSION A and the same in VERSION B

You add a comment or suppress that VULN' that has 'InstanceID- 33B44DD3471AC23153063D3EDBD9AFEF' in VERSION A

and

Later you upload VERSION B FPR that still contains the same VULN.
=======
However,
If you review the metadata for VULN - in 1st program in VERSION A
And
Also review the same VULN metadata in the Same Program in VERSION B

You will most likely discover the 'InstanceID' has changed for the VULN between Version A and Version B.

(In AWB right click the vuln to see analysis meta information)  

Example: Same XSS VULN in Same Program on Same Line# but different IIDs

'InstanceID- 33B44DD3471AC23153063D3EDBD9AFEF' was seen in VERSION A

'InstanceID- 33B44DD3471AC23153063D3EDBD96666' was seen for VERSION B

If you turn 'ON' AWB Suppressons and 'removed' you will see both VULNS (or make visible in SSC properties)

VERSION A has the Suppression and Comment info; and is flagged as REMOVED (FIXED InstanceID)

VERSION B has the same VULN with a NEW InstanceID value.

In general Instance ID generation is based on the following:
- Names of the variables involved in the issue
- Path name or Directory structure.
- If the issue involves function calls, the names of those functions - A canonical representation of the expression structure for expressions involved in the issue
- Names of enclosing function and class (if applicable) In the case of data flow issues, the instance ID is based on the source and sink nodes
- if the nodes between them change, the instance ID does not change.

In general, instance IDs are tolerant of line number, filename, and code structure changes, but are less tolerant of refactoring if function names, class names, or packages/namespaces are renamed.

Good news....
The release of SCA 19.10 (scheduled May 2019) has been updated to better tolerate -Path/Directory changes.

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.