(OO) Support Tip: New Python Executor server

1 Likes

New Python Executor server is introduced in HCMX 2023.05.

  • Switch from using an interpreter to using a new Python Executor that is based on a Python server with framework and library to evaluate all python-based pressions through REST API calls.
  • This feature will be applied on the Cloud Slang project and will affect all the workers within OO Central, OO RAS and OO Workflow Designer.
  • Each worker will have its own python executor server configured and running.
  • There will not be an option to use the python executor server from another worker
  • No changes are made to OO components or APIs.
  • No changes are made to the OO DB.
  • Main motivation for this architectural feature change is to dramatically improve performance for CloudSlang based flows which are affected by the slow evaluation of python-based expressions with the previous installed Python interpreter.
  • On average, the performance increase is about 95% with the selected 3rd party Python framework and server.

Labels:

Support Tip
Comment List
  • hiho,

    with 24.1 the reported disclaimer is "resolved": Known Problem OCTCR19F2018139 (microfocus.com)

    Regards Rene

  • Thanks for that.  I will definitely try it out.

  • hiho,

    the python-executor is the new default value. 

    But you can change it back by editing the wrapper.conf: https://docs.microfocus.com/doc/Operations_Orchestration/2023.05/ConfigurePythonExecutor

    So just replace "python-executor" with "python

    =wrapper.java.additional.47=-Dpython.expressionsEval=<INTERPRETER>

    Greetz Rene

  • When I upgraded the OO RASes from 2020.02 to 2023.05, I did not choose to use the python executor, so I thought it wouldn't be configured.  However, it created a "python-executor.properties" file and starts up.  What would be the cleanest way to stop this happening, please?  I think it may be causing issues with JVM memory errors.

    2023-11-20 23:43:30,564 [WrapperSimpleAppMain] (ExternalPythonExecutionEngine.java:49) INFO  - python timeout strategy: scheduled-executor
    2023-11-20 23:43:34,805 [WrapperSimpleAppMain] (PythonExecutorConfigurationDataServiceImpl.java:55) INFO  - Reading properties from: python-executor.properties
    2023-11-20 23:43:34,820 [WrapperSimpleAppMain] (PythonExecutorLifecycleManagerServiceImpl.java:171) INFO  - A request to start the Python Executor was sent
    2023-11-20 23:43:36,941 [WrapperSimpleAppMain] (PythonExecutorLifecycleManagerServiceImpl.java:213) INFO  - Starting Python Executor on port: 8002
    2023-11-20 23:43:36,957 [WrapperSimpleAppMain] (PythonExecutorLifecycleManagerServiceImpl.java:230) INFO  - Waiting to start
    2023-11-20 23:43:40,235 [WrapperSimpleAppMain] (PythonExecutorLifecycleManagerServiceImpl.java:234) INFO  - Python Executor was successfully started

  • hiho,

    even if the performance optimizations are very tempting, I would currently advise not using the new Python-Executor.

    Why is it currently not recommended to use the Python executor?

    After updating to OO 2023.05, we discovered that incorrect values ​​were returned as input or output when executing the same workflows at the same time. Specifically, we are using the Cloudslang function “cs_json_query”. The response from support: “Switch back to the classic Python-Interpreter”.

    And the problem here is not a small one: It's about data integrity in the ongoing workflows. Incorrect values ​​are used - depending on the purpose of the workflow, this can have serious consequences.

    We suspect the cause is the use of Uvicorn, but we don't yet know a real solution. However, the use of Uvicorn does not seem to be thread or transaction safe.

    Greetings Rene

Related
Recommended