Big news! The community will be moving to a new platform April 21. Read more.
Big news! The community will be moving to a new platform April 21. Read more.
Captain Captain
Captain
818 views

Fortify SCA unable to exclude python libraries from getting scanned

Hello Experts,

I am trying to scan python3 source files. I have installed python 3.6.8. Before running the scan, I enable python virtual environment by executing - 

./install/config-venv.sh install
source venv/bin/activate
VIRTUAL_ENV=/myworkspace/venv
++ export VIRTUAL_ENV

Here is the translation command:

sourceanalyzer -b Mypython -python-version 3 -python-path "/usr/lib64/python3.6:${VIRTUAL_ENV}/lib64/python3.6/site-packages:${VIRTUAL_ENV}/lib/python3.6/site-packages" "core/*.py" -exclude "core/xts.py" -exclude "venv/**/*.py" 

 

1. What is the below py file and why is it getting processed? 

Processing /myworkspace/user1/.fortify/sca19.1/build/Mypython/python-lib8976501320144073767/lib.py

2,  My source files are located in "core/*.py" and those are the only files I would like to scan. Any idea why SCA is scanning the python library file under /usr/lib64/python3.6?

Processing /usr/lib64/python3.6/abc.py

3. I have excluded a file "core/xts.py" in the above translation command, yet SCA keeps scanning it. Is there anything that I missed in the translation command?  The other exclude option works and "venv" is excluded. 

[error]: Unexpected exception while parsing file /myworkspace/core/xts.py
java.lang.NullPointerException: null
	at com.fortify.frontend.translator.nodes.STExpressionList.addChild(STExpressionList.java:60) ~[sca-frontend-19.1.2.0007.jar:?]
	at com.fortify.frontend.translator.python3.Python3Translator.visitTestlist_star_expr(Python3Translator.java:3584) ~[sca-frontend-19.1.2.0007.jar:?]

 

Greatly appreciate any suggestions/explanation on what is happening in this situation.

Thanks in advance.

 

 

 

0 Likes
6 Replies
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Do you see the file, /usr/lib64/python3.6/abc.py, running the following command,
eg
sourceanalyzer -b Mypython -show-files

Using a simple hello world example, eg print ('Hello World'), the following python library is "processed"

eg
sourceanalyzer -b helloworld -debug -verbose -python-version 3 python-example/helloword.py
Fortify Static Code Analyzer 20.1.0.0145 (using JRE 1.8.0_181)
Processing /root/.fortify/sca20.1/build/helloworld/python-lib8820943390916456136/lib.py
Processing /root/python-example/helloword.py

But SCA -show-files only shows what was scanned.

eg

sourceanalyzer -b helloworld -show-files
helloworld.py

Make sure the following warning message is not being displayed which could be the reason xts.py is not excluded.

eg

[warning]: No files were excluded as the file patterns: [path/filename.py] specified for -exclude option did not match any files.

Captain Captain
Captain

Thanks!! I do not see the python libraries in the result of the command "sourceanalyzer -b Mypython -show-files". I only see the intended files that needed to be scanned. Also, I do not see xts.py in the "-show-files" list, but it's strange that I should see it getting processed in the translation log with a nullpointerexception error.
0 Likes
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Is SCA starting under /myworkspace directory since the argument, -exclude "core/xts.py", indicates the core director is a sub folder of /myworkspace directory eg /myworkspace/core/xts.py"?

Can you try -exclude "**/core/**/xts.py"?

0 Likes
Cadet 3rd Class
Cadet 3rd Class

All, 

I am having similar issue, was there a solution to this?

```bash
sourceanalyzer --version
Fortify Static Code Analyzer 20.1.0.0158 (using JRE 1.8.0_181)

PYTHON_PATH="${CI_PROJECT_DIR}/venv/lib/python3.7/site-packages:${CI_PROJECT_DIR}/venv/lib/python3.9/site-packages"
 
sourceanalyzer -b ${CI_PIPELINE_ID} app/**/*.py -exclude "tests/*.py" -exclude "venv/**/*.py" -python-version 3 -python-path ${PYTHON_PATH}
```
where 
export CI_PIPELINE_ID=123456
export CI_PROJECT_DIR=/build

I have tried '**/venv/**/*.py', 'env/**'  and tried to move the position of the parameters around 
without success 
 
 
0 Likes
Cadet 3rd Class
Cadet 3rd Class

I get a bunch of these errors

[error]: Unexpected exception while parsing file /build/venv/lib/python3.9/site-packages/pydantic/types.py
java.lang.NullPointerException: null

and all the python files in packages are translated

...
venv/lib/python3.9/site-packages/urllib3/packages/six.py
venv/lib/python3.9/site-packages/urllib3/poolmanager.py
venv/lib/python3.9/site-packages/urllib3/request.py
venv/lib/python3.9/site-packages/urllib3/response.py
venv/lib/python3.9/site-packages/urllib3/util/__init__.py
venv/lib/python3.9/site-packages/urllib3/util/connection.py
venv/lib/python3.9/site-packages/urllib3/util/proxy.py
venv/lib/python3.9/site-packages/urllib3/util/queue.py
venv/lib/python3.9/site-packages/urllib3/util/request.py
venv/lib/python3.9/site-packages/urllib3/util/response.py
venv/lib/python3.9/site-packages/urllib3/util/retry.py
venv/lib/python3.9/site-packages/urllib3/util/ssl_.py
...
0 Likes
Captain Captain
Captain

The issues I was facing got resolved once I upgraded to SCA 20.1

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.