Highlighted
Absent Member.. Absent Member..
Absent Member..
907 views

Load Generator on Linux: Create Mutex Error [MsgId: MERR-10344]

I am encountering an error condition when attempting to connect my local LoadRunner Controller to a remote Load Generator agent running on a Linux host.

The error occurs after I have been using the agent on the remote host for several months.  That is, I am able to use the agent to run load tests successfully for a considerable amount of time (>150 days) until this error begins to occur.  Once the error condition occurs, it will persist until the host server is rebooted.  Shutting down and restarting the load generator software does not solve the problem.  Whatever the root cause, it appears a reboot of the host resolves the underlying condition.

Here is what I think is the primary log message of interest:

Error: Communication error: An error occurred while trying to create mutex. (sys error message - No space left on device) [MsgId: MERR-10344]

I have checked a few things:

  • All file systems have available space.
  • System real/swap memory is available.
  • I have attempted running strace against the process and while there are a few calls that return in error, a root cause does not stand out.

Here are a couple of host vitals:

Linux Host OS: RHEL 6.7
Load Generator Version: 12.02

Has anyone discovered a method to resolve this problem without having to reboot the host?

Thanks in advance. -Brian

Tags (4)
0 Likes
4 Replies
Highlighted
Frequent Contributor.
Frequent Contributor.

Re: Load Generator on Linux: Create Mutex Error [MsgId: MERR-10344]

Hello,

This looks relate to some shared memory locker used for agent's communication. We'll check.

Regards,

Bingle 

0 Likes
Highlighted
Absent Member.. Absent Member..
Absent Member..

Re: Load Generator on Linux: Create Mutex Error [MsgId: MERR-10344]

This definitely has the appearance of a shared memory problem.

Any luck tracking down the source?

Thanks, Brian

 

0 Likes
Highlighted
Frequent Contributor.
Frequent Contributor.

Re: Load Generator on Linux: Create Mutex Error [MsgId: MERR-10344]

Hello, 

If Vusers are stoped normally, there won't be such error, but if vusers were abnormally terminated, there are possibilities that some resources were not correctly freed, and when the error accumulate, there may be no space to create new Semaphores so the error message was give out.

I think you can try this workaround if don’t want to restart the LG.

Run m_daemon_setup stop

Run ipcs command to list the system wide resources.

Run ipcrm -s command to remove the unfreeed resources owned by LR (if we created a special user account to run the m_daemon_setup, it will be easy)

Run m_daemon_setup start.

Regards,

Bingle

0 Likes
Highlighted
Absent Member.. Absent Member..
Absent Member..

Re: Load Generator on Linux: Create Mutex Error [MsgId: MERR-10344]

Thanks for the follow-up, Bingle.

I ran the ipcs command and see that a couple semaphore arrays are in use by the LR user.

Next time we encounter the issue, I will attempt to free the resources using the method you've proposed.  I'll then post the results here.

Thanks again, Brian

 

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.