How does the MQ parameter of expiry work ? - Middleware News
Symptom Your messages are expiring from the MQ queue before your MQ applications have accessed them.
Fix
Following is a section from the IBM WMQ API reference on message expiry. To summarize, messages that expire continue to contribute to a queue's depth (message count) until they are browsed, an attempt is made to transport them to another queue is made, or an MQGET command is executed against them. When some action is taken against the expired message sitting on the queue, it is discarded at that point.
The only version of WMQ 5.3 that has the option of automatically removing expired messages from a queue (without having to browse, get, etc.) is the z/OS version.
Expiry (MQLONG)
Message lifetime.
This is a period of time expressed in tenths of a second, set by the application that
puts the message. The message becomes eligible to be discarded if it has not been
removed from the destination queue before this period of time elapses.
The value is decremented to reflect the time the message spends on the destination
queue, and also on any intermediate transmission queues if the put is to a remote
queue. It may also be decremented by message channel agents to reflect
transmission times, if these are significant. Likewise, an application forwarding this
message to another queue might decrement the value if necessary, if it has retained
the message for a significant time. However, the expiration time is treated as
approximate, and the value need not be decremented to reflect small time
intervals.
When the message is retrieved by an application using the MQGET call, the Expiry
field represents the amount of the original expiry time that still remains.
After a message?s expiry time has elapsed, it becomes eligible to be discarded by
the queue manager. In the current implementations, the message is discarded when
a browse or nonbrowse MQGET call occurs that would have returned the message
had it not already expired. For example, a nonbrowse MQGET call with the
MatchOptions field in MQGMO set to MQMO_NONE reading from a FIFO ordered
queue will cause all the expired messages to be discarded up to the first unexpired
message. With a priority ordered queue, the same call will discard expired
messages of higher priority and messages of an equal priority that arrived on the
queue before the first unexpired message.
A message that has expired is never returned to an application (either by a browse
or a non-browse MQGET call), so the value in the Expiry field of the message
descriptor after a successful MQGET call is either greater than zero, or the special
value MQEI_UNLIMITED.
If a message is put on a remote queue, the message may expire (and be discarded)
whilst it is on an intermediate transmission queue, before the message reaches the
destination queue.
A report is generated when an expired message is discarded, if the message
specified one of the MQRO_EXPIRATION_* report options. If none of these
options is specified, no such report is generated; the message is assumed to be no
longer relevant after this time period (perhaps because a later message has
superseded it).
Any other program that discards messages based on expiry time must also send an
appropriate report message if one was requested.
Notes:
1. If a message is put with an Expiry time of zero, the MQPUT or MQPUT1 call
fails with reason code MQRC_EXPIRY_ERROR; no report message is generated
in this case.
2. Since a message whose expiry time has elapsed may not actually be discarded
until later, there may be messages on a queue that have passed their expiry
Chapter 10. MQMD ? Message descriptor 151
time, and which are not therefore eligible for retrieval. These messages
nevertheless count towards the number of messages on the queue for all
purposes, including depth triggering.
3. An expiration report is generated, if requested, when the message is actually
discarded, not when it becomes eligible for discarding.
4. Discarding of an expired message, and the generation of an expiration report if
requested, are never part of the application?s unit of work, even if the message
was scheduled for discarding as a result of an MQGET call operating within a
unit of work.
5. If a nearly-expired message is retrieved by an MQGET call within a unit of
work, and the unit of work is subsequently backed out, the message may
become eligible to be discarded before it can be retrieved again.
6. If a nearly-expired message is locked by an MQGET call with MQGMO_LOCK,
the message may become eligible to be discarded before it can be retrieved by
an MQGET call with MQGMO_MSG_UNDER_CURSOR; reason code
MQRC_NO_MSG_UNDER_CURSOR is returned on this subsequent MQGET
call if that happens.
7. When a request message with an expiry time greater than zero is retrieved, the
application can take one of the following actions when it sends the reply
message:
v Copy the remaining expiry time from the request message to the reply
message.
v Set the expiry time in the reply message to an explicit value greater than
zero.
v Set the expiry time in the reply message to MQEI_UNLIMITED.
The action to take depends on the design of the application suite. However, the
default action for putting messages to a dead-letter (undelivered-message)
queue should be to preserve the remaining expiry time of the message, and to
continue to decrement it.
8. Trigger messages are always generated with MQEI_UNLIMITED.
9. A message (normally on a transmission queue) which has a Format name of
MQFMT_XMIT_Q_HEADER has a second message descriptor within the
MQXQH. It therefore has two Expiry fields associated with it. The following
additional points should be noted in this case:
v When an application puts a message on a remote queue, the queue manager
places the message initially on a local transmission queue, and prefixes the
application message data with an MQXQH structure. The queue manager
sets the values of the two Expiry fields to be the same as that specified by
the application.
If an application puts a message directly on a local transmission queue, the
message data must already begin with an MQXQH structure, and the format
name must be MQFMT_XMIT_Q_HEADER (but the queue manager does not
enforce this). In this case the application need not set the values of these two
Expiry fields to be the same. (The queue manager does not check that the
Expiry field within the MQXQH contains a valid value, or even that the
message data is long enough to include it.)
v When a message with a Format name of MQFMT_XMIT_Q_HEADER is
retrieved from a queue (whether this is a normal or a transmission queue),
the queue manager decrements both these Expiry fields with the time spent
waiting on the queue. No error is raised if the message data is not long
enough to include the Expiry field in the MQXQH.
MQMD ? Expiry field
152 Application Programming Reference
v The queue manager uses the Expiry field in the separate message descriptor
(that is, not the one in the message descriptor embedded within the MQXQH
structure) to test whether the message is eligible for discarding.
v If the initial values of the two Expiry fields were different, it is therefore
possible for the Expiry time in the separate message descriptor when the
message is retrieved to be greater than zero (so the message is not eligible for
discarding), while the time according to the Expiry field in the MQXQH has
elapsed. In this case the Expiry field in the MQXQH is set to zero.
The following special value is recognized:
MQEI_UNLIMITED
Unlimited lifetime.
The message has an unlimited expiration time.
On VSE/ESA, the value of Expiry must be MQEI_UNLIMITED.
This is an output field for the MQGET call, and an input field for the MQPUT and
MQPUT1 calls. The initial value of this field is MQEI_UNLIMITED.
Expired messages on z/OS
On WebSphere MQ for z/OS, messages that have expired are discarded by the
next appropriate MQGET call. However, if no such call occurs, the expired
message is not discarded, and, for some queues, a large number of expired
messages can accumulate. To remedy this, you can set the queue manager to scan
queues periodically and discard expired messages on one or more queues.
There are two ways of doing this:
Periodic scan
You can specify a period using the EXPRYINT (expiry interval) queue
manager attribute. Each time the expiry interval is reached, the queue
manager looks for candidate queues that are worth scanning to discard
expired messages.
The queue manager maintains information about the expired messages on
each queue, and knows whether a scan for expired messages is
worthwhile. So, only a selection of queues is scanned at any time.
Shared queues are scanned only by one queue manager in a queue-sharing
group. Generally, it is the first queue manager to restart, or the first to
have EXPRYINT set. If this queue manager terminates, another queue
manager in the queue-sharing group takes over the queue scanning. Set the
expiry interval value for all queue managers within a queue-sharing group
to the same value.
Explicit request
Issue the REFRESH QMGR TYPE(EXPIRY) command, specifying the queue
or queues that you want scanned.
MQMD ? Expiry field
Note
The xCoupler (RA56-ESA Release 1.2 and greater) can set the EXPIRY for the MQ messages in 2 locations.
1. On an individual TRIGGER level (on the trigger edit screen under Extended Attributes)
2. On an individual TRANSPORT level (on the transport edit screen under Extended Attributes)
Symptom Your messages are expiring from the MQ queue before your MQ applications have accessed them.
Fix
Following is a section from the IBM WMQ API reference on message expiry. To summarize, messages that expire continue to contribute to a queue's depth (message count) until they are browsed, an attempt is made to transport them to another queue is made, or an MQGET command is executed against them. When some action is taken against the expired message sitting on the queue, it is discarded at that point.
The only version of WMQ 5.3 that has the option of automatically removing expired messages from a queue (without having to browse, get, etc.) is the z/OS version.
Expiry (MQLONG)
Message lifetime.
This is a period of time expressed in tenths of a second, set by the application that
puts the message. The message becomes eligible to be discarded if it has not been
removed from the destination queue before this period of time elapses.
The value is decremented to reflect the time the message spends on the destination
queue, and also on any intermediate transmission queues if the put is to a remote
queue. It may also be decremented by message channel agents to reflect
transmission times, if these are significant. Likewise, an application forwarding this
message to another queue might decrement the value if necessary, if it has retained
the message for a significant time. However, the expiration time is treated as
approximate, and the value need not be decremented to reflect small time
intervals.
When the message is retrieved by an application using the MQGET call, the Expiry
field represents the amount of the original expiry time that still remains.
After a message?s expiry time has elapsed, it becomes eligible to be discarded by
the queue manager. In the current implementations, the message is discarded when
a browse or nonbrowse MQGET call occurs that would have returned the message
had it not already expired. For example, a nonbrowse MQGET call with the
MatchOptions field in MQGMO set to MQMO_NONE reading from a FIFO ordered
queue will cause all the expired messages to be discarded up to the first unexpired
message. With a priority ordered queue, the same call will discard expired
messages of higher priority and messages of an equal priority that arrived on the
queue before the first unexpired message.
A message that has expired is never returned to an application (either by a browse
or a non-browse MQGET call), so the value in the Expiry field of the message
descriptor after a successful MQGET call is either greater than zero, or the special
value MQEI_UNLIMITED.
If a message is put on a remote queue, the message may expire (and be discarded)
whilst it is on an intermediate transmission queue, before the message reaches the
destination queue.
A report is generated when an expired message is discarded, if the message
specified one of the MQRO_EXPIRATION_* report options. If none of these
options is specified, no such report is generated; the message is assumed to be no
longer relevant after this time period (perhaps because a later message has
superseded it).
Any other program that discards messages based on expiry time must also send an
appropriate report message if one was requested.
Notes:
1. If a message is put with an Expiry time of zero, the MQPUT or MQPUT1 call
fails with reason code MQRC_EXPIRY_ERROR; no report message is generated
in this case.
2. Since a message whose expiry time has elapsed may not actually be discarded
until later, there may be messages on a queue that have passed their expiry
Chapter 10. MQMD ? Message descriptor 151
time, and which are not therefore eligible for retrieval. These messages
nevertheless count towards the number of messages on the queue for all
purposes, including depth triggering.
3. An expiration report is generated, if requested, when the message is actually
discarded, not when it becomes eligible for discarding.
4. Discarding of an expired message, and the generation of an expiration report if
requested, are never part of the application?s unit of work, even if the message
was scheduled for discarding as a result of an MQGET call operating within a
unit of work.
5. If a nearly-expired message is retrieved by an MQGET call within a unit of
work, and the unit of work is subsequently backed out, the message may
become eligible to be discarded before it can be retrieved again.
6. If a nearly-expired message is locked by an MQGET call with MQGMO_LOCK,
the message may become eligible to be discarded before it can be retrieved by
an MQGET call with MQGMO_MSG_UNDER_CURSOR; reason code
MQRC_NO_MSG_UNDER_CURSOR is returned on this subsequent MQGET
call if that happens.
7. When a request message with an expiry time greater than zero is retrieved, the
application can take one of the following actions when it sends the reply
message:
v Copy the remaining expiry time from the request message to the reply
message.
v Set the expiry time in the reply message to an explicit value greater than
zero.
v Set the expiry time in the reply message to MQEI_UNLIMITED.
The action to take depends on the design of the application suite. However, the
default action for putting messages to a dead-letter (undelivered-message)
queue should be to preserve the remaining expiry time of the message, and to
continue to decrement it.
8. Trigger messages are always generated with MQEI_UNLIMITED.
9. A message (normally on a transmission queue) which has a Format name of
MQFMT_XMIT_Q_HEADER has a second message descriptor within the
MQXQH. It therefore has two Expiry fields associated with it. The following
additional points should be noted in this case:
v When an application puts a message on a remote queue, the queue manager
places the message initially on a local transmission queue, and prefixes the
application message data with an MQXQH structure. The queue manager
sets the values of the two Expiry fields to be the same as that specified by
the application.
If an application puts a message directly on a local transmission queue, the
message data must already begin with an MQXQH structure, and the format
name must be MQFMT_XMIT_Q_HEADER (but the queue manager does not
enforce this). In this case the application need not set the values of these two
Expiry fields to be the same. (The queue manager does not check that the
Expiry field within the MQXQH contains a valid value, or even that the
message data is long enough to include it.)
v When a message with a Format name of MQFMT_XMIT_Q_HEADER is
retrieved from a queue (whether this is a normal or a transmission queue),
the queue manager decrements both these Expiry fields with the time spent
waiting on the queue. No error is raised if the message data is not long
enough to include the Expiry field in the MQXQH.
MQMD ? Expiry field
152 Application Programming Reference
v The queue manager uses the Expiry field in the separate message descriptor
(that is, not the one in the message descriptor embedded within the MQXQH
structure) to test whether the message is eligible for discarding.
v If the initial values of the two Expiry fields were different, it is therefore
possible for the Expiry time in the separate message descriptor when the
message is retrieved to be greater than zero (so the message is not eligible for
discarding), while the time according to the Expiry field in the MQXQH has
elapsed. In this case the Expiry field in the MQXQH is set to zero.
The following special value is recognized:
MQEI_UNLIMITED
Unlimited lifetime.
The message has an unlimited expiration time.
On VSE/ESA, the value of Expiry must be MQEI_UNLIMITED.
This is an output field for the MQGET call, and an input field for the MQPUT and
MQPUT1 calls. The initial value of this field is MQEI_UNLIMITED.
Expired messages on z/OS
On WebSphere MQ for z/OS, messages that have expired are discarded by the
next appropriate MQGET call. However, if no such call occurs, the expired
message is not discarded, and, for some queues, a large number of expired
messages can accumulate. To remedy this, you can set the queue manager to scan
queues periodically and discard expired messages on one or more queues.
There are two ways of doing this:
Periodic scan
You can specify a period using the EXPRYINT (expiry interval) queue
manager attribute. Each time the expiry interval is reached, the queue
manager looks for candidate queues that are worth scanning to discard
expired messages.
The queue manager maintains information about the expired messages on
each queue, and knows whether a scan for expired messages is
worthwhile. So, only a selection of queues is scanned at any time.
Shared queues are scanned only by one queue manager in a queue-sharing
group. Generally, it is the first queue manager to restart, or the first to
have EXPRYINT set. If this queue manager terminates, another queue
manager in the queue-sharing group takes over the queue scanning. Set the
expiry interval value for all queue managers within a queue-sharing group
to the same value.
Explicit request
Issue the REFRESH QMGR TYPE(EXPIRY) command, specifying the queue
or queues that you want scanned.
MQMD ? Expiry field
Note
The xCoupler (RA56-ESA Release 1.2 and greater) can set the EXPIRY for the MQ messages in 2 locations.
1. On an individual TRIGGER level (on the trigger edit screen under Extended Attributes)
2. On an individual TRANSPORT level (on the transport edit screen under Extended Attributes)
Howdy! I just would like to give an enormous thumbs up for the nice data you've right here on this post. I will be coming again to your blog for extra soon. slots for real money
ReplyDelete