Highlighted
Absent Member.
Absent Member.
1848 views

Call to WINAPI "memcmp" gives exception

Jump to solution

Hi,

In one of our Cobol programs (compiled for .NET) we have a call to the WINAPI memcmp function. This call is working properly and gives the correct results.

As we are now experimenting with multi-threading our application, we ran into a problem with this call. The second thread that executes throws an exception on the WINAPI "memcmp" call:

System.ArgumentException: Duplicate dynamic module name within an assembly.
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at System.Reflection.Emit.AssemblyBuilderData.CheckNameConflict(String strNewModuleName)
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at System.Reflection.Emit.AssemblyBuilder.DefineDynamicModuleInternalNoLock(String name, Boolean emitSymbolInfo, StackCrawlMark& stackMark)
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at System.Reflection.Emit.AssemblyBuilder.DefineDynamicModuleInternal(String name, Boolean emitSymbolInfo, StackCrawlMark& stackMark)
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at System.Reflection.Emit.AssemblyBuilder.DefineDynamicModule(String name)
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at MicroFocus.COBOL.Program.Win32.NativeProcedurePointer.GetMethodInfo(CobolCallConvention callConvention, Object[] parms)
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at MicroFocus.COBOL.Program.ProcedurePointer.call(UInt32 callConvention, Object[] parms)
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at MicroFocus.COBOL.Program.Control.CallReturningObject(UInt32 callConvention, String program, Object[] parameters, IObjectControl pgInstance)
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at MicroFocus.COBOL.Program.Control.Call(UInt32 callConvention, String program, Object[] parameters, IObjectControl pgInstance)
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at MEMCMP-SALSTRK.MEMCMP_SALSTRK(Reference SALSTRKGEG, Reference SALSTRKGEG-BEWAAR, Reference MEMCMP-RETURNCODE) in D:\Workspace2012\Gemal\ENGINE\1_ONTWIKKEL_R11\source\Cobol\MEMCMP-SALSTRK.cbl:line 29
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at (Object , Object[] )
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at MicroFocus.COBOL.Program.ProcedurePointer.call(UInt32 callConvention, Object[] parms)
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at MicroFocus.COBOL.Program.Control.CallReturningObject(UInt32 callConvention, String program, Object[] parameters, IObjectControl pgInstance)
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at MicroFocus.COBOL.Program.Control.Call(UInt32 callConvention, String program, Object[] parameters, IObjectControl pgInstance)
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at D01M4095._MF_PERFORM_4_1(_MF_LCTYPE_1& _MF_LCDATA) in D:\Workspace2012\Gemal\ENGINE\1_ONTWIKKEL_R11\source\Cobol\D01M4095.cbl:line 521
ERROR 2017-03-06 13:45:54 - D01WN00-1      :    at D01M4095.D01M4095(Reference REC-1, Reference LBCOP03, Reference INSTGEG-IG, Reference RESULTTAB-RE, Reference NFTAB-NF, Reference UVPER-TAB, Reference SALSTRKGEG, Reference SOCIALEVERZVLDN-SV, Reference INSTGEG-PERIODE-IG, Reference WERKNEMER-INVOERTOTAALTABEL-P, Reference

 

The call to memcmp is in a separate module:

IDENTIFICATION DIVISION.

$set ilpinvoke"msvcrt"

PROGRAM-ID. MEMCMP-SALSTRK.

AUTHOR. MARK DE VOS.

DATE-WRITTEN. MAART 2017.

*********************************************************************************************

* MODULE MEMCMP-SALSTRK: COMPAREN VAN DE GROEPSVELDEN SALSTRKRESTAB-RS EN SALSTRKGEG-BEWAAR

* VIA EEN CALL NAAR DE WINAPI METHODE memcmp

*********************************************************************************************

ENVIRONMENT DIVISION.

CONFIGURATION SECTION.

SPECIAL-NAMES.

CALL-CONVENTION 74 IS WINAPI.

DATA DIVISION.

WORKING-STORAGE SECTION.

LINKAGE SECTION.

COPY D01CP117.

COPY D01028CP.

01 MEMCMP-RETURNCODE PIC 9(9).

PROCEDURE DIVISION USING SALSTRKGEG

SALSTRKGEG-BEWAAR

MEMCMP-RETURNCODE.

HOOFD SECTION.

HOOFD-01.

CALL "cob32api.dll".

*

MOVE ZERO TO MEMCMP-RETURNCODE.

*

CALL WINAPI "memcmp" USING SALSTRKRESTAB-RS

SALSTRKGEG-BEWAAR

LENGTH-OF-SALSTRKRESTAB-RS

RETURNING MEMCMP-RETURNCODE.

*

INVOKE type Logger::Info(string::Format("MEMCMP-RETURNCODE: {0}", MEMCMP-RETURNCODE)).

HOOFD-99.

EXIT PROGRAM.

 

By the way, the reason that we are using memcmp is that the regular Cobol comparison of two very large groupfields is performing very, very slow in .Net.

0 Likes
1 Solution

Accepted Solutions
Highlighted
Absent Member.
Absent Member.

RE: Call to WINAPI "memcmp" gives exception

Jump to solution
I have found a solution for my problem.

I have put the call to "memcmp" in a static Cobol class method and now it works fine in a multithreaded setting.

Here's the new code:

$set ilpinvoke"msvcrt"
CLASS-ID Raet.Gemal.Engine.Cobol.Classes.MemoryCompare PUBLIC STATIC.
WORKING-STORAGE SECTION.
METHOD-ID CompareSalstrkgeg PUBLIC STATIC.
LOCAL-STORAGE SECTION.
01 LENGTH-OF-SALSTRKRESTAB-RS TYPE Int32.
LINKAGE SECTION.
COPY D01CP117.
COPY D01028CP.
01 MEMCMP-RETURNCODE TYPE Int32.
/
PROCEDURE DIVISION USING BY REFERENCE SALSTRKGEG
BY REFERENCE SALSTRKGEG-BEWAAR
BY REFERENCE MEMCMP-RETURNCODE.
/
HOOFD SECTION.
HOOFD-01.
SET LENGTH-OF-SALSTRKRESTAB-RS TO LENGTH OF SALSTRKRESTAB-RS.
*
MOVE ZERO TO MEMCMP-RETURNCODE.
*
TRY
CALL "memcmp" USING BY REFERENCE SALSTRKRESTAB-RS
BY REFERENCE SALSTRKGEG-BEWAAR
BY VALUE LENGTH-OF-SALSTRKRESTAB-RS
RETURNING MEMCMP-RETURNCODE
CATCH
E AS TYPE System.Exception
INVOKE type Logger::Error("Call to memcmp has raised an exception", E)
RAISE
FINALLY
CONTINUE
END-TRY.
HOOFD-99.
EXIT METHOD.
END METHOD.
/
END CLASS.


And in one of the calling modules:

INVOKE TYPE Raet.Gemal.Engine.Cobol.Classes.MemoryCompare::CompareSalstrkgeg(SALSTRKGEG, SALSTRKSPIRAALTAB-RS, MEMCMP-SALSTRKGEG-RETURNCODE).

View solution in original post

0 Likes
8 Replies
Highlighted
Micro Focus Expert
Micro Focus Expert

RE: Call to WINAPI "memcmp" gives exception

Jump to solution
Have you reported the performance problem with the COBOL group items compare in a support incident? I would recommend this as a first step so that we can take a look at the problem in-house.

The solution that you are attempting here has a few problems and is not a safe one for multi-threaded code unless you are using COBOL RunUnits.

Can you please tell me what type of application this is and how are you starting your threads?

BTW, memcmp is not a WINAPI call but is instead a function in the C/C++ library.
It should actually look like:
CALL "memcmp" USING SALSTRKRESTAB-RS SALSTRKGEG-BEWAAR by value LENGTH-OF-SALSTRKRESTAB-RS

You also should not be calling "cob32api.dll" in managed code.

Can you give me a full example with copybooks and threading code?
0 Likes
Highlighted
Micro Focus Expert
Micro Focus Expert

RE: Call to WINAPI "memcmp" gives exception

Jump to solution
memcmp is __cdecl, not WINAPI (__stdcall). So calling it using WINAPI is already a potential problem. So I believe you need call-convention 0, not 74.

It doesn't seem like that would be responsible for this issue, which sounds more like both threads are trying to generate the wrapper method for the memcmp call. I'd suggest trying to avoid the issue by refactoring, for example by having one thread at startup call a method that sets a procedure-pointer to 'entry "memcmp"' and after that calls through the procedure-pointer.

If that fails, you should raise a support incident.

That said, I don't recommend using memcmp. For one thing, if there's any filler in the group item, its value isn't guaranteed, so you may get false negatives.

Even if you're sure that you want a bit-for-bit comparison, P/Invoke is not the way I'd solve this, personally. Unfortunately the method I'd recommend relies on an undocumented feature, so I can't describe it here. I'm not sure what the official recommendation from the compiler team would be.
0 Likes
Highlighted
Micro Focus Expert
Micro Focus Expert

RE: Call to WINAPI "memcmp" gives exception

Jump to solution
Chris - I'll just note that a significant performance difference is to be expected. Even for C# (or IL generated from any other CLR language), people who have looked into this report something on the order of 50x difference for comparing byte arrays with pure managed code versus native code. It's possible to get performance comparable to native for comparing large byte arrays by using unsafe pointers and manually unrolled loops, but that requires the two objects be bit-for-bit identical, which may not be the case with COBOL group items that are logically equivalent.

Good point about COBOL RunUnits; I'd forgotten about that.
0 Likes
Highlighted
Absent Member.
Absent Member.

RE: Call to WINAPI "memcmp" gives exception

Jump to solution
Hi Chris and Michael,

Thanks for your reactions.

The module that calls "memcmp" is part of a large payroll application. It's a mixed C# - Cobol application.

So the code that starts the threads is in C# and looks like this:

public override void Run(JobRunProperties jobrunproperties, RunStatus status)
{
...
...

try
{
RunParallel(jobrunproperties, status);

if (Threads > 1)
{
MergeThreadFiles();
}
}
catch (AggregateException ae)
{
foreach (var ex in ae.InnerExceptions)
{
Logger.Log(LogLevel.Error, ex.Message, ex);
}

throw new Exception("Error(s) during Cobol multithreaded execution");
}
...
...
}

private void RunParallel(JobRunProperties jobrunproperties, RunStatus status)
{
ConcurrentQueue<Exception> exceptions = new ConcurrentQueue<Exception>();
IEnumerable<int> seq = Enumerable.Range(1, Threads);
ParallelOptions paralleloptions = new ParallelOptions { MaxDegreeOfParallelism = jobrunproperties.MaxThreads };

Parallel.ForEach(seq, paralleloptions, (currentThread) =>
{
try
{
RunModule(currentThread, status);
}
catch (Exception e)
{
exceptions.Enqueue(e);
}
});

if (exceptions.Count > 0)
{
throw new AggregateException(exceptions);
}
}

private void RunModule(int thread, RunStatus status)
{
SetFileIoVariables(thread);

IDictionary<string, string> threadParameters = new Dictionary<string, string>(Parameters);
threadParameters.Add("THREADID", thread.ToString());

RunUnit myRunUnit = new RunUnit();

try
{
int threadStatus = myRunUnit.Call("GEMAL-ENTRY", new StepRunProperties(Name, Id, threadParameters), DatabaseConnectionSettings);

lock (status)
{
if (threadStatus > status.ReturnCode)
{
status.ReturnCode = threadStatus;
}
}
}
catch (Exception ex)
{
throw new Exception(string.Format("Fout tijdens verwerken Cobol \n{0}", ex.Message), ex);
}
finally
{
try
{
myRunUnit.StopRun();
}
catch
{
}
}
}

So, each thread creates an instance of the MicroFocus.COBOL.RuntimeServices.RunUnit class and calls the Cobol module "GEMAL-ENTRY". This module calls dozens of other Cobol modules. In several of those modules a call to the module that uses the "memcmp" function is programmed.

In the module that uses the "memcmp" function I removed the CALL-CONVENTION declaration and I changed the call to "memcmp". So, it now looks like this:

CALL "memcmp" USING SALSTRKRESTAB-RS,
SALSTRKGEG-BEWAAR
LENGTH-OF-SALSTRKRESTAB-RS
RETURNING MEMCMP-RETURNCODE.

When I run the application single-threaded everything works fine and the results are ok. But running with more threads I still get the same exception:

System.ArgumentException: Duplicate dynamic module name within an assembly.
0 Likes
Highlighted
Absent Member.
Absent Member.

RE: Call to WINAPI "memcmp" gives exception

Jump to solution
I'd like to add another remark.

I noticed that different threads have already been using the call to "memcmp" succesfully before the exception gets thrown. To me this seems very odd.
0 Likes
Highlighted
Absent Member.
Absent Member.

RE: Call to WINAPI "memcmp" gives exception

Jump to solution
The last remark I made was wrong. Only the first thread that starts executing uses the memcmp successfully. All the other threads throw an exception.
0 Likes
Highlighted
Micro Focus Expert
Micro Focus Expert

RE: Call to WINAPI "memcmp" gives exception

Jump to solution
The last parameter to memcmp should be passed by value. You should also remove the call to "cob32api".

Unfortunately, calls to native code are not protected within a RunUnit. You may try surrounding the call with a sync block but the sync object will have to be outside of the RunUnit:

program-id. gsync.
working-storage section.
procedure division.
sync on type SharedObj::static-sync-point
call "memcmp" ...
end-sync
.
end program.



class-id. SharedObj public.
01 static-sync-point string value "notimportant"
static public initialize only.
end class.
0 Likes
Highlighted
Absent Member.
Absent Member.

RE: Call to WINAPI "memcmp" gives exception

Jump to solution
I have found a solution for my problem.

I have put the call to "memcmp" in a static Cobol class method and now it works fine in a multithreaded setting.

Here's the new code:

$set ilpinvoke"msvcrt"
CLASS-ID Raet.Gemal.Engine.Cobol.Classes.MemoryCompare PUBLIC STATIC.
WORKING-STORAGE SECTION.
METHOD-ID CompareSalstrkgeg PUBLIC STATIC.
LOCAL-STORAGE SECTION.
01 LENGTH-OF-SALSTRKRESTAB-RS TYPE Int32.
LINKAGE SECTION.
COPY D01CP117.
COPY D01028CP.
01 MEMCMP-RETURNCODE TYPE Int32.
/
PROCEDURE DIVISION USING BY REFERENCE SALSTRKGEG
BY REFERENCE SALSTRKGEG-BEWAAR
BY REFERENCE MEMCMP-RETURNCODE.
/
HOOFD SECTION.
HOOFD-01.
SET LENGTH-OF-SALSTRKRESTAB-RS TO LENGTH OF SALSTRKRESTAB-RS.
*
MOVE ZERO TO MEMCMP-RETURNCODE.
*
TRY
CALL "memcmp" USING BY REFERENCE SALSTRKRESTAB-RS
BY REFERENCE SALSTRKGEG-BEWAAR
BY VALUE LENGTH-OF-SALSTRKRESTAB-RS
RETURNING MEMCMP-RETURNCODE
CATCH
E AS TYPE System.Exception
INVOKE type Logger::Error("Call to memcmp has raised an exception", E)
RAISE
FINALLY
CONTINUE
END-TRY.
HOOFD-99.
EXIT METHOD.
END METHOD.
/
END CLASS.


And in one of the calling modules:

INVOKE TYPE Raet.Gemal.Engine.Cobol.Classes.MemoryCompare::CompareSalstrkgeg(SALSTRKGEG, SALSTRKSPIRAALTAB-RS, MEMCMP-SALSTRKGEG-RETURNCODE).

View solution in original post

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.