Intermittent File Not Found

I'm running 10.80 and in my flows there are operations that reference files such as a powershell operation.

I created a flow to illustrate the problem.  I have a powershell operation that runs a hello world script.  If you look at the attached log you will see that the first two attempts running the operations resulted in an error.  Each time the operation fails I have it retry .. Failure --> Do Nothing --> Retry.  Finally after the third attempt the powershell operation completes (finds the ps1 file).  I'm not sure what is causing java to not be able to locate files on the OO Central server.  (There is only one central/RAS server in the enviornment).

 

Steps undertaken by the flow run "Debug[PSTest]" with id 1430552115

FLOW: Flow data missing.

Description: Flow data missing.
Operation path: Flow data missing.
Start time: 09-Jul-2018 at 11:42:57 CDT
End time: 09-Jul-2018 at 11:43:26 CDT
Run time [s]: 29.187
Response: success
Bound inputs: {}
Raw results: {Result=}
User: 85c8bdf6-d13a-48e5-866f-f4d58f9b5699\nathan.kippen
Triggering source: studio
Flow ROI: 0.0


1. PowerShell Script[OPERATION]

Description:
Current Flow: PSTest
Current Flow ID: 439742f4-d010-4f8b-a47f-a4f5656491db
User: ESRM\kippen
Start time: 09-Jul-2018 at 11:42:57 CDT
End time: 09-Jul-2018 at 11:43:01 CDT
Run time[s]: 4.124
Worker ID: 7666dcf8-8579-4f48-8e64-ff07b790de77
Worker Group: RAS_Operator_Path
Execution ID: 1430552115
Bound inputs: {scriptTimeout=null, enablePiping=False, runspaceID=null, modules=null, snapins=null, script=c:\scripts\hello.ps1, cmdlet=null, returnTable=False, delimiter=null, colDelimiter=null, rowDelimiter=null}
Transition name: failure
Transition Message: failure
Primary result: 1
Step results: {}
Raw Results: {exception=dotNETWebActions_PowerShell.PowerShellExceptions.ExtraCommandException: Could not execute powershell command.At line:1 char:1
c:\scripts\hello.ps1
~~~~~~~~~~~~~~~~~~~~ ---> System.Management.Automation.CommandNotFoundException: The term 'c:\scripts\hello.ps1' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
at System.Management.Automation.CommandDiscovery.LookupCommandInfo(String commandName, CommandTypes commandTypes, SearchResolutionOptions searchResolutionOptions, CommandOrigin commandOrigin, ExecutionContext context)
at System.Management.Automation.CommandDiscovery.LookupCommandProcessor(String commandName, CommandOrigin commandOrigin, Nullable`1 useLocalScope)
at System.Management.Automation.ExecutionContext.CreateCommand(String command, Boolean dotSource)
at System.Management.Automation.PipelineOps.AddCommand(PipelineProcessor pipe, CommandParameterInternal[] commandElements, CommandBaseAst commandBaseAst, CommandRedirection[] redirections, ExecutionContext context)
at System.Management.Automation.PipelineOps.InvokePipeline(Object input, Boolean ignoreInput, CommandParameterInternal[][] pipeElements, CommandBaseAst[] pipeElementAsts, CommandRedirection[][] commandRedirections, FunctionContext funcContext)
at System.Management.Automation.Interpreter.ActionCallInstruction`6.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
--- End of inner exception stack trace ---
at PowerShellV2.PSRunspaceV2.RunCommand[T](PSCommand command)
at PowerShellV2.PSRunspaceV2.RunCommand(PSCommand command)
at PowerShellV2.PSRunspaceV2.RunScript(String scriptText)
at dotNETWebActions_PowerShell.IActions.PowerShellBaseAction.ExecutePowerShell(ActionRequest req, ISession s, IActionRegistry reg), returnCode=1, stdout=null, returnResult=Could not execute powershell command.At line:1 char:1
c:\scripts\hello.ps1
~~~~~~~~~~~~~~~~~~~~, sessionId=new session, isSessionOn=False, script=c:\scripts\hello.ps1, Result=1}
ROI: 0.0
Errors: {}

2. Do Nothing[OPERATION]

Description:
Current Flow: PSTest
Current Flow ID: 439742f4-d010-4f8b-a47f-a4f5656491db
User: ESRM\nathan.kippen
Start time: 09-Jul-2018 at 11:43:01 CDT
End time: 09-Jul-2018 at 11:43:01 CDT
Run time[s]: 0.078
Worker ID: 7666dcf8-8579-4f48-8e64-ff07b790de77
Worker Group: RAS_Operator_Path
Execution ID: 1430552115
Bound inputs: {field1=null, field2=null, field3=null, field4=null}
Transition name: success
Transition Message: success
Primary result:
Step results: {}
Raw Results: {response=success, returnResult=null, TimedOut=false, FailureMessage=null, Result=null}
ROI: 0.0
Errors: {}

3. PowerShell Script[OPERATION]

Description:
Current Flow: PSTest
Current Flow ID: 439742f4-d010-4f8b-a47f-a4f5656491db
User: ESRM\kippen
Start time: 09-Jul-2018 at 11:43:01 CDT
End time: 09-Jul-2018 at 11:43:04 CDT
Run time[s]: 2.995
Worker ID: 7666dcf8-8579-4f48-8e64-ff07b790de77
Worker Group: RAS_Operator_Path
Execution ID: 1430552115
Bound inputs: {scriptTimeout=null, enablePiping=False, runspaceID=null, modules=null, snapins=null, script=c:\scripts\hello.ps1, cmdlet=null, returnTable=False, delimiter=null, colDelimiter=null, rowDelimiter=null}
Transition name: failure
Transition Message: failure
Primary result: 1
Step results: {}
Raw Results: {exception=dotNETWebActions_PowerShell.PowerShellExceptions.ExtraCommandException: Could not execute powershell command.At line:1 char:1
c:\scripts\hello.ps1
~~~~~~~~~~~~~~~~~~~~ ---> System.Management.Automation.CommandNotFoundException: The term 'c:\scripts\hello.ps1' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
at System.Management.Automation.CommandDiscovery.LookupCommandInfo(String commandName, CommandTypes commandTypes, SearchResolutionOptions searchResolutionOptions, CommandOrigin commandOrigin, ExecutionContext context)
at System.Management.Automation.CommandDiscovery.LookupCommandProcessor(String commandName, CommandOrigin commandOrigin, Nullable`1 useLocalScope)
at System.Management.Automation.ExecutionContext.CreateCommand(String command, Boolean dotSource)
at System.Management.Automation.PipelineOps.AddCommand(PipelineProcessor pipe, CommandParameterInternal[] commandElements, CommandBaseAst commandBaseAst, CommandRedirection[] redirections, ExecutionContext context)
at System.Management.Automation.PipelineOps.InvokePipeline(Object input, Boolean ignoreInput, CommandParameterInternal[][] pipeElements, CommandBaseAst[] pipeElementAsts, CommandRedirection[][] commandRedirections, FunctionContext funcContext)
at System.Management.Automation.Interpreter.ActionCallInstruction`6.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
--- End of inner exception stack trace ---
at PowerShellV2.PSRunspaceV2.RunCommand[T](PSCommand command)
at PowerShellV2.PSRunspaceV2.RunCommand(PSCommand command)
at PowerShellV2.PSRunspaceV2.RunScript(String scriptText)
at dotNETWebActions_PowerShell.IActions.PowerShellBaseAction.ExecutePowerShell(ActionRequest req, ISession s, IActionRegistry reg), returnCode=1, stdout=null, returnResult=Could not execute powershell command.At line:1 char:1
c:\scripts\hello.ps1
~~~~~~~~~~~~~~~~~~~~, sessionId=new session, isSessionOn=False, script=c:\scripts\hello.ps1, Result=1}
ROI: 0.0
Errors: {}

4. Do Nothing[OPERATION]

Description:
Current Flow: PSTest
Current Flow ID: 439742f4-d010-4f8b-a47f-a4f5656491db
User: ESRM\kippen
Start time: 09-Jul-2018 at 11:43:04 CDT
End time: 09-Jul-2018 at 11:43:04 CDT
Run time[s]: 0.078
Worker ID: 7666dcf8-8579-4f48-8e64-ff07b790de77
Worker Group: RAS_Operator_Path
Execution ID: 1430552115
Bound inputs: {field1=null, field2=null, field3=null, field4=null}
Transition name: success
Transition Message: success
Primary result:
Step results: {}
Raw Results: {response=success, returnResult=null, TimedOut=false, FailureMessage=null, Result=null}
ROI: 0.0
Errors: {}

5. PowerShell Script[OPERATION]

Description:
Current Flow: PSTest
Current Flow ID: 439742f4-d010-4f8b-a47f-a4f5656491db
User: ESRM\kippen
Start time: 09-Jul-2018 at 11:43:04 CDT
End time: 09-Jul-2018 at 11:43:26 CDT
Run time[s]: 21.81
Worker ID: 7f628deb-1f49-44d9-9e2a-ff438895bd0c
Worker Group: RAS_Operator_Path
Execution ID: 1430552115
Bound inputs: {scriptTimeout=null, enablePiping=False, runspaceID=null, modules=null, snapins=null, script=c:\scripts\hello.ps1, cmdlet=null, returnTable=False, delimiter=null, colDelimiter=null, rowDelimiter=null}
Transition name: success
Transition Message: success
Primary result: 0
Step results: {}
Raw Results: {returnCode=0, warnings=null, returnResult=Hello World
, sessionId=new session, script=c:\scripts\hello.ps1, objectsCount=1, Result=0}
ROI: 0.0
Errors: {}

6. Resolved : success[RETURN_STEP]

Description:
Current Flow: PSTest
Current Flow ID: 439742f4-d010-4f8b-a47f-a4f5656491db
User: ESRM\kippen
Start time: 09-Jul-2018 at 11:43:26 CDT
End time: 09-Jul-2018 at 11:43:26 CDT
Run time[s]: 0.01
Worker ID:
Worker Group:
Execution ID: 1430552115
Bound inputs: {}
Transition name:
Transition Message:
Primary result:
Step results: {}
Raw Results: {}
ROI:
Errors: {}

  • Verified Answer

    Central has an internal worker
    RAS is an explicit worker

    Your operation is failing on the node below
    Worker ID: 7666dcf8-8579-4f48-8e64-ff07b790de77
    Worker Group: RAS_Operator_Path

    But succeeding on 
    Worker ID: 7f628deb-1f49-44d9-9e2a-ff438895bd0c
    Worker Group: RAS_Operator_Path

    You can see which worker id is which node by looking in ("System Configuration" -> "Topology") -- if the column isn't visible add it (icon next to clear filter on the header bar)

    You can 
    1) copy the script to both nodes
    2) disable the worker on the failing node
    3) assign the successful worker to a new "worker group" and explicitly call the Execute PS Script operation on it
    --- (Add operation input on PS Script Execute "overrideNRAS" and set the value to equal the new worker group name)