Absent Member.
Absent Member.
5465 views

Software Security Assurance with Higher-order functions using HP Fortify

Higher-order features such as lambdas exist ubiquitously in web applications and frameworks. They make development easier, but at a cost of added complexity and exposure to high risk vulns and attacks. However, statically ruling out such vulnerabilities is theoretical and practically challenging, especially when high-order functions and complex control-flow collide with opaque, dynamic data structures such as objects.

 

This talk aims to provide an easy-to-understand explanation of higher-order function and the difficulties involved in assessing it. We’ll include a brief report on the how HP Fortify Static Code Analyzer handles higher-order analysis and our plans for future improvements. Note: Content focused on a technical-level viewer.

0 Likes
7 Replies
Absent Member.
Absent Member.

Since what version is the CESK* Abstract machine part of the SCA?

0 Likes
Absent Member.
Absent Member.

Hello,


This was introduced with 4.2 but was a primitive HOA model without tuning. In 4.3 we introduced pushdown higher-order analysis.


Thank you for watching the webinar!

0 Likes
Absent Member.
Absent Member.

For some additional reading:

David Van Horn and Matthew Might. "Abstracting Abstract Machines." International Conference on Functional Programming 2010 (ICFP 2010). Baltimore, Maryland. September, 2010. pages 51--62.
Paper: [pdf] [errata]

David Van Horn and Matthew Might. "Systematic abstraction of abstract machines." Journal of Functional Programming. 2012.
Paper: [pdf] [arXiv]


Pushdown control-flow analysis and its application:

Pruning, Pushdown Exception-Flow Analysis
Shuying Liang, Weibin Sun, Matthew Might, Andrew Keep, David Van Horn
SCAM 2014, The 14th IEEE International Working Conference on Source Code Analysis and Manipulation. (acceptance rate: 31.7%).
PDF

Sound and Precise Malware Analysis for Android via Pushdown Reachability and Entry-Point Saturation
Shuying Liang, Andy Keep, Matthew Might, David Van Horn,
3rd Annual ACM CCS workshop on Security and Privacy in SmartPhones and Mobile Devices (SPSM'13), Berlin, Germany, 2013. (acceptance rate: 24%).
PDF

0 Likes
Vice Admiral
Vice Admiral

Sorry I did not watch the presentation but at 14:00 I caught the statement that the higher-order analysis may get into an infinite loop.

I found this post only after seeing an 80KB server-side Javascript code whose analysis does not complete after waiting ~10 minutes on a 16GB laptop (sourceanalyzer 20.1.2's autoheap resulting in allocating more than 14GB of RAM, according to my MacOS Activity Monitor), even when using -Dcom.fortify.sca.Phase0HigherOrder.Timeout.Hard=60 .

Is it OK to assume that Javascript and Typescript should be removed from the higher order analysis list, for all practical uses?  (I have no idea what Phase0 is).

com.fortify.sca.Phase0HigherOrder.Languages

Default: python,ruby,swift,javascript,typescript

https://www.microfocus.com/documentation/fortify-static-code-analyzer-and-tools/2010/SCA_Help_20.1.2/index.htm#ConfigProperties/SCAProps.htm?Highlight=higher

 

0 Likes
Vice Admiral
Vice Admiral

SCA 20.2.0 still spins on the file.

Attaching the .nst and the log.

 

0 Likes
Vice Admiral
Vice Admiral

A Javascript React file took forever in my trying to scan it, so one reduced version of it threw this after half hour,

$ /Applications/Fortify/Fortify_SCA_and_Apps_20.2.1/jre/bin/java -jar /Applications/Fortify/Fortify_SCA_and_Apps_20.2.1/Core/lib/exe/sca-exe.jar -Dcom.fortify.sca.follow.imports=false -Dcom.fortify.sca.hoa.Enable=true -rules /Applications/Fortify/Fortify_SCA_and_Apps_20.2.1/Core/config/rules -scan e.js
[error]: There is not enough memory available to complete analysis. For details on making more memory available, please consult the user manual.
java.lang.OutOfMemoryError: Java heap space
at com.esotericsoftware.kryo.util.IdentityObjectIntMap.resize(IdentityObjectIntMap.java:541) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.util.IdentityObjectIntMap.push(IdentityObjectIntMap.java:277) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.util.IdentityObjectIntMap.put(IdentityObjectIntMap.java:161) ~[kryo-4.0.0.jar:?]
at com.fortify.sca.util.serialization.SCABackEndSerializer$SCAKryoMapReferenceResolver.addWrittenObject(SCABackEndSerializer.java:107) ~[fortify-sca-20.2.1.0010.jar:?]
at com.esotericsoftware.kryo.Kryo.writeReferenceOrNull(Kryo.java:681) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:570) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:505) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:505) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:505) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:505) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651) ~[kryo-4.0.0.jar:?]
at com.fortify.sca.util.serialization.SCABackEndSerializer$8.write(SCABackEndSerializer.java:582) ~[fortify-sca-20.2.1.0010.jar:?]
at com.fortify.sca.util.serialization.SCABackEndSerializer$8.write(SCABackEndSerializer.java:575) ~[fortify-sca-20.2.1.0010.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:505) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:505) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651) ~[kryo-4.0.0.jar:?]
at com.fortify.sca.util.serialization.SCABackEndSerializer$8.write(SCABackEndSerializer.java:582) ~[fortify-sca-20.2.1.0010.jar:?]
at com.fortify.sca.util.serialization.SCABackEndSerializer$8.write(SCABackEndSerializer.java:575) ~[fortify-sca-20.2.1.0010.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:505) ~[kryo-4.0.0.jar:?]
[error]: Unexpected exception: java.lang.BootstrapMethodError: call site initialization exception
java.lang.BootstrapMethodError: call site initialization exception
at java.lang.invoke.CallSite.makeSite(CallSite.java:341) ~[?:1.8.0_181]
at java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307) ~[?:1.8.0_181]
at java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297) ~[?:1.8.0_181]
at com.fortify.sca.analyzer.constants.ConstantPropagator.propagationPasses(ConstantPropagator.java:365) ~[fortify-sca-20.2.1.0010.jar:?]
at com.fortify.sca.analyzer.constants.ConstantPropagator.analyze(ConstantPropagator.java:180) ~[fortify-sca-20.2.1.0010.jar:?]
at com.fortify.sca.backend.BackEnd.doPhaseZero(BackEnd.java:1586) ~[fortify-sca-20.2.1.0010.jar:?]
at com.fortify.sca.backend.BackEnd.analyze(BackEnd.java:828) ~[fortify-sca-20.2.1.0010.jar:?]
at com.fortify.sca.Main$Sourceanalyzer.run(Main.java:667) ~[fortify-sca-20.2.1.0010.jar:?]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at jdk.internal.org.objectweb.asm.ClassWriter.newUTF8(ClassWriter.java:1122) ~[?:1.8.0_181]
at jdk.internal.org.objectweb.asm.MethodWriter.getSize(MethodWriter.java:2128) ~[?:1.8.0_181]
at jdk.internal.org.objectweb.asm.ClassWriter.toByteArray(ClassWriter.java:856) ~[?:1.8.0_181]
at java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:310) ~[?:1.8.0_181]
at java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:194) ~[?:1.8.0_181]
at java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:304) ~[?:1.8.0_181]
at java.lang.invoke.LambdaForm$DMH/1521118594.invokeStatic_L6_L(LambdaForm$DMH) ~[?:?]
at java.lang.invoke.LambdaForm$BMH/978841414.reinvoke(LambdaForm$BMH) ~[?:?]
at java.lang.invoke.LambdaForm$MH/697960108.invoke_MT(LambdaForm$MH) ~[?:?]
at java.lang.invoke.CallSite.makeSite(CallSite.java:302) ~[?:1.8.0_181]
... 7 more

 

Another, even smaller, version of the file took more than 20 minutes (almost a second per line of code) and threw an exception along with some findings.  I am attaching the file in the hopes that someone could suggest a way to limit the scan efforts or resolve a bug in the "high order analysis" mode enabling which causes so much trouble.  (Disabling HOA shortens the time and memory consumption but loses simple XSS findings).

 

$ /Applications/Fortify/Fortify_SCA_and_Apps_20.2.1/jre/bin/java -jar /Applications/Fortify/Fortify_SCA_and_Apps_20.2.1/Core/lib/exe/sca-exe.jar -Dcom.fortify.sca.follow.imports=false -Dcom.fortify.sca.hoa.Enable=true -rules /Applications/Fortify/Fortify_SCA_and_Apps_20.2.1/Core/config/rules -scan e.js
[error]: There is not enough memory available to complete analysis. For details on making more memory available, please consult the user manual.
java.lang.OutOfMemoryError: Java heap space
at com.ergy.fset.FHashMap$FHMIterator.<init>(FHashMap.java:1891) ~[fset-1.0.4.jar:?]
at com.ergy.fset.FHashMap.iterator(FHashMap.java:359) ~[fset-1.0.4.jar:?]
at com.fortify.sca.util.serialization.SCABackEndSerializer$4.write(SCABackEndSerializer.java:493) ~[fortify-sca-20.2.1.0010.jar:?]
at com.fortify.sca.util.serialization.SCABackEndSerializer$4.write(SCABackEndSerializer.java:488) ~[fortify-sca-20.2.1.0010.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:505) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651) ~[kryo-4.0.0.jar:?]
at com.fortify.sca.util.serialization.SCABackEndSerializer$8.write(SCABackEndSerializer.java:582) ~[fortify-sca-20.2.1.0010.jar:?]
at com.fortify.sca.util.serialization.SCABackEndSerializer$8.write(SCABackEndSerializer.java:575) ~[fortify-sca-20.2.1.0010.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:505) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:505) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651) ~[kryo-4.0.0.jar:?]
at com.fortify.sca.util.serialization.SCABackEndSerializer$8.write(SCABackEndSerializer.java:582) ~[fortify-sca-20.2.1.0010.jar:?]
at com.fortify.sca.util.serialization.SCABackEndSerializer$8.write(SCABackEndSerializer.java:575) ~[fortify-sca-20.2.1.0010.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:505) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:505) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651) ~[kryo-4.0.0.jar:?]
at com.fortify.sca.util.serialization.SCABackEndSerializer$8.write(SCABackEndSerializer.java:582) ~[fortify-sca-20.2.1.0010.jar:?]
at com.fortify.sca.util.serialization.SCABackEndSerializer$8.write(SCABackEndSerializer.java:575) ~[fortify-sca-20.2.1.0010.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:505) ~[kryo-4.0.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575) ~[kryo-4.0.0.jar:?]

[/Users/latypil/tmp/snippet-485]

[3F1C00C435A6378E3829E3FD6CD56464 : critical : Key Management : Hardcoded Encryption Key : structural ]
e.js(578)
Field: anonymous~object.key [e.js(577)]

[3F1C00C435A6378E3829E3FD6CD56465 : critical : Key Management : Hardcoded Encryption Key : structural ]
e.js(599)
Field: anonymous~object.key [e.js(598)]

[3F1C00C435A6378E3829E3FD6CD56466 : critical : Key Management : Hardcoded Encryption Key : structural ]
e.js(716)
Field: anonymous~object.key [e.js(715)]

[3F1C00C435A6378E3829E3FD6CD56467 : critical : Key Management : Hardcoded Encryption Key : structural ]
e.js(810)
Field: anonymous~object.key [e.js(809)]

 

0 Likes
Vice Admiral
Vice Admiral

Giving more (12GB) memory and some garbage collection options tuned the analysis.

$ time /Applications/Fortify/Fortify_SCA_and_Apps_20.2.1/jre/bin/java -Dcom.fortify.sca.MultithreadedAnalysis=false -Dcom.fortify.sca.ThreadCount=1 -XX:+CMSClassUnloadingEnabled -XX:+UseParallelGC -Dcom.fortify.sca.follow.imports=false -Dcom.fortify.sca.hoa.Enable=true -Xmx12G -jar /Applications/Fortify/Fortify_SCA_and_Apps_20.2.1/Core/lib/exe/sca-exe.jar -rules /Applications/Fortify/Fortify_SCA_and_Apps_20.2.1/Core/config/rules -scan e.js

[/Users/latypil/tmp/snippet-485]

[3F1C00C435A6378E3829E3FD6CD56464 : critical : Key Management : Hardcoded Encryption Key : structural ]
    e.js(578)
    Field: anonymous~object.key [e.js(577)]

[3F1C00C435A6378E3829E3FD6CD56465 : critical : Key Management : Hardcoded Encryption Key : structural ]
    e.js(599)
    Field: anonymous~object.key [e.js(598)]

[3F1C00C435A6378E3829E3FD6CD56466 : critical : Key Management : Hardcoded Encryption Key : structural ]
    e.js(716)
    Field: anonymous~object.key [e.js(715)]

[3F1C00C435A6378E3829E3FD6CD56467 : critical : Key Management : Hardcoded Encryption Key : structural ]
    e.js(810)
    Field: anonymous~object.key [e.js(809)]

real	4m33.778s
user	11m54.908s
sys	0m38.893s

 

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.