Fortify Cloud Scan Issues

[Amrityam_Rout]

Hello Everyone,

We are using Fortify cloud scan 19.10 .For Controller and Sensors(5 sensors) we are using Windows server 2016 and client is Jenkins/GoCD hosted in  linux environment.

What we observed is, in every   10/15 scans 3/4 no scans are getting scan faulted issue.

I am attaching the Sensor and Controller logs for reference.

2020-01-30 02:29:44,127 [INFO] com.fortify.cloud.cli.command.WorkerCommand - Controller responded with SCAN_JOB Job ID: 45bf0ace-a746-4b3d-b6fd-97becdef5d35
2020-01-30 02:30:00,954 [ERROR] com.fortify.cloud.cli.command.WorkerCommand - Job processing failed with exception: Cannot download file 'job.zip' from controller: I/O error on GET request for "http://XX.XX.XX.XX:8080/cloud-ctrl/rest/worker/v1/request/45bf0ace-a746-4b3d-b6fd-97becdef5d35": Read timed out; nested exception is java.net.SocketTimeoutException: Read timed out
2020-01-30 02:30:00,954 [INFO] com.fortify.cloud.cli.command.WorkerCommand - Sending job state FAULTED to controller
2020-01-30 02:30:00,969 [INFO] com.fortify.cloud.cli.command.WorkerCommand - Controller responded: OK
2020-01-30 02:33:27,279 [INFO] com.fortify.cloud.cli.command.WorkerCommand - Controller responded with SCAN_JOB Job ID: 541deccb-7dc6-4daa-b9da-21d94c41e8a0
2020-01-30 02:33:41,646 [ERROR] com.fortify.cloud.cli.command.WorkerCommand - Job processing failed with exception: Cannot download file 'job.zip' from controller: I/O error on GET request for "http://XX.XX.XX.XX:8080/cloud-ctrl/rest/worker/v1/request/541deccb-7dc6-4daa-b9da-21d94c41e8a0": Read timed out; nested exception is java.net.SocketTimeoutException: Read timed out
2020-01-30 02:33:41,646 [INFO] com.fortify.cloud.cli.command.WorkerCommand - Sending job state FAULTED to controller
2020-01-30 02:33:41,662 [INFO] com.fortify.cloud.cli.command.WorkerCommand - Controller responded: OK
2020-01-30 02:47:11,320 [INFO] com.fortify.cloud.cli.command.WorkerCommand - Controller responded with SCAN_JOB Job ID: 17407109-3285-4a4d-a646-1b342219edaa
2020-01-30 02:47:25,664 [ERROR] com.fortify.cloud.cli.command.WorkerCommand - Job processing failed with exception: Cannot download file 'job.zip' from controller: I/O error on GET request for "http://XX.XXX.XX.XX:8080/cloud-ctrl/rest/worker/v1/request/17407109-3285-4a4d-a646-1b342219edaa": Read timed out; nested exception is java.net.SocketTimeoutException: Read timed out
2020-01-30 02:47:25,664 [INFO] com.fortify.cloud.cli.command.WorkerCommand - Sending job state FAULTED to controller
2020-01-30 02:47:25,680 [INFO] com.fortify.cloud.cli.command.WorkerCommand - Controller responded: OK

Controller Logs:

2020-01-30 08:21:13,092 [DEBUG] [10.190.74.37 POST /cloud-ctrl/rest/worker/v1/request] com.fortify.cloud.ctrl.web.controller.CoreController - WORKER_ACTIVITY:10.190.74.37:workrequest
2020-01-30 08:21:13,380 [DEBUG] [10.190.74.50 GET /cloud-ctrl/rest/worker/v1/status/e47c9add-8053-4cbd-a5b1-dd9a9cc9a4d3] com.fortify.cloud.ctrl.spring.security.WorkerAuthenticationFilter - Worker authorized [10.190.74.50/34e021e9-2f41-4ce4-b3b8-bfc98a2e57b3] with fortify-worker header
2020-01-30 08:21:13,380 [DEBUG] [10.190.74.50 GET /cloud-ctrl/rest/worker/v1/status/e47c9add-8053-4cbd-a5b1-dd9a9cc9a4d3] com.fortify.cloud.ctrl.web.controller.CoreController - WORKER_ACTIVITY:10.190.74.50:get_status
2020-01-30 08:21:13,380 [DEBUG] [10.190.74.50 POST /cloud-ctrl/rest/worker/v1/status/e47c9add-8053-4cbd-a5b1-dd9a9cc9a4d3] com.fortify.cloud.ctrl.spring.security.WorkerAuthenticationFilter - Worker authorized [10.190.74.50/34e021e9-2f41-4ce4-b3b8-bfc98a2e57b3] with fortify-worker header
2020-01-30 08:21:13,380 [INFO] [10.190.74.50 POST /cloud-ctrl/rest/worker/v1/status/e47c9add-8053-4cbd-a5b1-dd9a9cc9a4d3] com.fortify.cloud.ctrl.web.controller.CoreController - controller entered WORK STATUS update handler
2020-01-30 08:21:13,380 [INFO] [10.190.74.50 POST /cloud-ctrl/rest/worker/v1/status/e47c9add-8053-4cbd-a5b1-dd9a9cc9a4d3] com.fortify.cloud.ctrl.web.controller.CoreController - WORKER_ACTIVITY:10.190.74.50:update_status
2020-01-30 08:21:13,380 [INFO] [10.190.74.50 POST /cloud-ctrl/rest/worker/v1/status/e47c9add-8053-4cbd-a5b1-dd9a9cc9a4d3] com.fortify.cloud.ctrl.service.JobManagerServiceImpl - State changed for Job e47c9add-8053-4cbd-a5b1-dd9a9cc9a4d3 from QUEUED to FAULTED
2020-01-30 08:21:13,396 [INFO] com.fortify.cloud.ctrl.service.JobManagerServiceImpl - SSC upload state changed for Job e47c9add-8053-4cbd-a5b1-dd9a9cc9a4d3 from QUEUED to CANCELED

Let me know if anyone has faced this kind of issue?

@ebell @Tejesh Chandra @HansEnders 

[ebell]

It appears for some reason there is a communications issue between the client and controller causing the socket to time out - SocketTimeoutException: Read timed out. Have you ruled out any environmental issues? Take a look at the Windows Event logs (Application and System) to see if there is anything of importance around those times.