Pause Flow OO Task with Wakeupscheduler

Idea ID 2748345

Pause Flow OO Task with Wakeupscheduler

Brief Description

Currently when you would like to have a flow waiting for a bit longer period of time you would use a Sleep action in the Flow. the Big disadvantage with this is that the Flow is blocking an Execution Thread on the respective RAS. on a loaded environment with potentially varying load patterns with a lot of flows that use the Sleep action this can result is slowing down environment as Flows are fighting for remaining Threads that are not in Sleep.

suggest to introduce a Task in OO that basically puts the Flow into Paused or similar state.

the advanatage of Paused state is the Flow is removed from the RAS/execution handling.

at the same time wakeup time or pause time in seconds must be given to the Task.

At the specified Time the Flow will be resumed.

Just increasing Execution threads is not an option as other loadscenarios would then overload the RAS due to the uneven CPU/RAM/Execution thread balance.

 

Benefits / Value -Explain why is this important? (When used, by which role, and benefits / value)-

This allows OO to be used in larger scale environments where certain longer running Flows need a pause state that does not consume system resources while in the paused state.

Supporting scalability and also proper use of resources.

also probably any autoscaling function of OO in CDF can be impacted by above issue as the pure CPU load on a RAS does not increase when all Execution threads are in sleep mode.

Design details

-Explain how would you like this idea designed / implemented

an OO Central Background thread watching Wakeuptimer for every centrally paused flow.

additionally that could be also used for UI pausing (specifying a wakeup time)

additionally an OO Action/Task that can be used in a flow to put Flow into Paused mode while still finishing this task. (critical is here correct timing between Central putting Flow to pause and the step finishing). only using OO APi within the Flow to put itself to pause is not sufficent as you cant rely that OO pauses the Flow exactly after this step which means that additional steps would be executed afterwards without pause which are only intended to be executed after wakeup.

This could be implemented as an extensible pause solution supporting similar problem scenarios like "SA Wait for Job"

Tags (1)
8 Comments
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

"only using OO APi within the Flow to put itself to pause is not sufficent as you cant rely that OO pauses the Flow exactly after this step which means that additional steps would be executed afterwards without pause which are only intended to be executed after wakeup".

OO should pause the flow exactly after the step that is executed when receiving the call is finished. If this is not the case, please let us know of a specific situation in which this doesn't happen.

Michael_it Honored Contributor.
Honored Contributor.
I did run another sequence of tests in Load conditions of OO. Where probably high percentage of flows are going to Pause exactly after the flow it self was requesting the pause.
But there are definitely occasions where after successful pause command a few additional steps are executed.
I was taking this as works as designed.
Situation:
2 node Clustered OO Central
1 Remote RAS configured with 10 execution Threads.
~ 150 Flows in parallel to be partially executed on this RAS.
Pause call will be issued within the flow by the RAS mentioned above
Subsequent Steps executed at the same ras

I can provide Logs If required. Should I open case on this one
Niels Walta Respected Contributor.
Respected Contributor.

@Michael_it wrote:

Brief Description

Currently when you would like to have a flow waiting for a bit longer period of time you would use a Sleep action in the Flow.


Is there any reason why you wouldn't then opt to redesign the flow and make use of the scheduling mechanics in OO? We've been using that quite successfully for a large number of our processes, which require long waiting times (weeks!).

This involved either making the flows "restartable" (either skipping over steps or making steps superfluous on subsequent runs) or splitting up the process (into multiple "follow-up" flows). Using unique identifiers in the schedule names, we can easily trace back which schedule belongs to which flow run.

And ultimately, this means we can keep the "Paused" state as an identifier that something abnormal is going on with the run; e.g. someone has manually halted the run or workers are missing.

Eager to hear your thoughts on this, in regard to your situation!

Michael_it Honored Contributor.
Honored Contributor.

This methode we also use in some areas but it is much much less adviceable from our PoV.

1. you have to have extra logic in the flows to restart at the relevant step

2. you create extra load in the system to process up to the resume step

3. you can not see ONE WORKFLOW result for analysis. (you need to search together/have an extra system that matches the individual runs)

4. OO scheduler limited and unmanageable at scale (e.g. you can not permission individual Schedules ... its an all you can eat permission. maintenance management of OO with scheduler (disable/enable) is painful)

we disucssed this option for a while and it has much more disadvanategs than the other proposal

We want to have OO to design Workflow that suites our needs and not building workflows around the Gaps of the Product. the whole intention is to have one workflow/run and not 20 single Runs.

I would see using Scheduler and Spliting rather as a workaround then a solution.

as depicted in the Idea i agree with you on the Paused state.... i suggested to "Flow into Paused or similar state"

so this probably would be good to have the states separated. depending on development effort and impact i rather would use paused when i would have to wait for solution longer to introduce another State

at least this is our PoV 🙂

Michael_it Honored Contributor.
Honored Contributor.

ps.: and just to add i am also talking about wait for e.g. 5 minutes... not a week. when you have high parallelity of Flows that do use Sleep for 5 minutes you soon have a dead OO due to the above described issue. and for 5 minutes waiting scheduling/splitting is to much overhead

Niels Walta Respected Contributor.
Respected Contributor.

@Michael_it wrote:

ps.: and just to add i am also talking about wait for e.g. 5 minutes... not a week. when you have high parallelity of Flows that do use Sleep for 5 minutes you soon have a dead OO due to the above described issue. and for 5 minutes waiting scheduling/splitting is to much overhead

Thanks for the elaboration, Michael! When put into context like that your request makes a lot more sense to me. So much so, that you've got my vote!


 

Micro Focus Frequent Contributor
Micro Focus Frequent Contributor
Status changed to: Waiting for Votes

I will move this Idea in Waiting for Votes and see how it evolves.

In the meantime we'll investigate the pause flow behavior that you've described.

Michael_it Honored Contributor.
Honored Contributor.

additional design consideration:

Current action to pause a flow is based on OO API and requires full specification of Login details. usually you can use logged in user as input to the operation. but i am not sure if this works in all scearios (Scheduled Runs, API triggered, Handed over to different user).

in case a new operation e.g. "Hibernate Myself until" is implemented maybe it would be good to do this with internal credentials easing the use of the operation.

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.