WebSphere MQ client connection failures due to resource shortages - Middleware News

WebSphere MQ client connection failures due to resource shortages
Technote (troubleshooting) - Middleware News

Problem(Abstract)
Your WebSphere MQ client suddenly fails when attempting to connect to your MQ server.

Symptom
You receive many FDCs for the AMQRMPPA and RUNMQLSR processes. The FDCs indicate a shortage of system resources.

xecP_E_NO_RESOURCE
rrcE_CREATE_THREAD_ERROR
xecL_E_TOO_MANY_HANDLES
rc 1450 (NO_SYSTEM_RESOURCES) from WriteFile
rc 12 (ENOMEM = insufficient system resources)

Cause
Recent changes to your Security Exit program caused the resource problem.

Diagnosing the problem
The following errors were reported in FDC files on the server:

Probe Id :- XC035007
Component :- xcsCreateThread
Process Name :- C:\Program Files\IBM\WebSphere MQ\bin\amqzlaa0.exe
Major Errorcode :- xecP_E_NO_RESOURCE
Comment1 :- Not enough space

Probe Id :- ZT049001
Component :- zutPreReadMachineIniFile
Process Name :- C:\Program Files\IBM\WebSphere MQ\bin\amqmsrvn.exe
Major Errorcode :- xecF_E_UNEXPECTED_RC
Minor Errorcode :- xecU_W_KEY_NOT_FOUND

Probe Id :- ZS266000
Component :- lpiObtainQMDetails
Process Name :- C:\Program Files\IBM\WebSphere MQ\bin\amqmsrvn.exe
Major Errorcode :- xecF_E_UNEXPECTED_RC
Minor Errorcode :- MQRC_UNEXPECTED_ERROR

Probe Id :- ZC000040
Component :- zcpCreatePipe
Process Name :- C:\Program Files\IBM\WebSphere MQ\bin\amqzxma0.exe
Major Errorcode :- xecF_E_UNEXPECTED_RC
Minor Errorcode :- xecL_E_TOO_MANY_HANDLES

Probe Id :- ZX005025
Component :- zxcProcessChildren
Process Name :- C:\Program Files\IBM\WebSphere MQ\bin\amqzxma0.exe
Major Errorcode :- zrcX_PROCESS_MISSING

Probe Id :- XY195001
Component :- xstSemaphoreRequest
Process Name :- C:\Program Files\IBM\WebSphere MQ\bin\amqxssvn.exe
Major Errorcode :- xecL_E_TOO_MANY_HANDLES
Comment1 :- Failed to create Event semaphore:Global\mqm.qmgrs.MAIL1!STATION
Comment2 :- Not enough storage is available to process this command.

Probe Id :- XC381009
Component :- xcsDisplayMessageForSubpool
Process Name :- C:\Program Files\IBM\WebSphere MQ\bin\RUNMQLSR.exe
Major Errorcode :- xecF_E_UNEXPECTED_SYSTEM_RC
Comment1 :- WinNT error 1450 from WriteFile.
Comment2 :- Insufficient system resources exist

Refer to the WebSphere MQ Messages manual, chapter 8, for a list of MQ reason codes and their meanings.

Resolving the problem
Make corrections to your security exit so that it does not leak system resources like handles and memory.



AMQ9508 RUNMQSC Command Fails
Technote (troubleshooting)

Problem(Abstract)

You make a user a member of the mqm group. You logon with that uid, test with the RUNMQSC command and fail with the following:

$ RUNMQSC
Starting MQSeries Commands.
AMQ9508: Program cannot connect to the queue manager.
No MQSC commands read.
No commands have a syntax error.
All valid MQSC commands were processed.

Resolving the problem
WebSphere MQ caches group membership for all users at startup. You must restart the queue manager to pick up changes in group membership.



AMQ9524 and AMQ9508 reason code MQRC_SECURITY_ERROR (2063)
Technote (troubleshooting)


Problem(Abstract)

Your queue manager fails to connect to another WebSphere® MQ Solaris queue manager. The connection fails with: AMQ9524 - queue manager not available.


Symptom
Error logs shows:

AMQ9508: Program cannot connect to the queue manager.
EXPLANATION:
The connection attempt to queue manager 'SDQA41' failed with reason code
2063 MQRC_SECURITY_ERROR.

AMQ9509: Program cannot open queue manager object.
EXPLANATION:
The attempt to open either the queue or queue manager object 'ICTXXT01'
queue manager 'SDQA41' failed with reason code 2085.

Cause

The issue is with the userid "mqm" being created LOCALLY. The Solaris box was using NIS+. The security error was due to a conflict between the local userid and NIS+ userid.

Resolving the problem
Delete the local userid "mqm".


Additional information
2063 0x0000080f MQRC_SECURITY_ERROR
2085 0x00000825 MQRC_UNKNOWN_OBJECT_NAME

FDC's report the following:

MQSeries First Failure Symptom Report
| =====================================
| Host Name :- XXXXXXXX (SunOS 5.6)
| PIDS :- 5765B75
| LVLS :- 520
| Product Long Name :- MQSeries for Sun Solaris 2 (Sparc)
| Vendor :- IBM
| Probe Id :- ZF048015
| Application Name :- MQM
| Component :- zfu_as_searchprincipallist
| Build Date :- Nov 7 2000
| CMVC level :- p000-L001106
| Build Type :- IKAP - (Production)
| UserID :- 00011001 (root)
| Program Name :- amqzlaa0_nd
| Process :- 00014376
| Thread :- 00000003
| QueueManager :- SDQA41
| Major Errorcode :- krcE_UNEXPECTED_ERROR
| Minor Errorcode :- OK
| Probe Type :- INCORROUT
| Probe Severity :- 2
| Probe Description :- AMQ6125: An internal MQSeries error has occurred.
|
+--------------------------------------------------------------

MQM Function Stack
zlaMainThread
zlaProcessMessage
zlaProcessMQIRequest
zlaMQCONN
zsqMQCONN
kpiMQCONN
kqiAuthorityChecks
gpiCheckObjectAuthority
zfu_as_checkobjectauthority
zfu_as_calculateauthority
zfu_as_searchprincipallist
xcsFFST