We are running very busy environmenet with IDAA being frequently refreshed by loader with increamnts of inserted records. In addition, we are using the DB2 automated housekeeping approach with the DSNACCOX stored proc implmentation to manage and trigger all required housekeeping/maintenance actions on objects in DB2. There is often a conflict and failure of the load process observed (daily basis, up to circa 100 failures) when the UNLOAD part of idaa load hits the "utility incompatible" error due to reorg being run on the same partition which needs to be unloaded and piped into the IDAA box. etails below.
How it is impacting us - we are running a very busy environment which is highly dependent on the contents of IDAA tables which are frequently loaded. During the critical business periods we frequently insert and update a lot of key tables and at the same time, we need to keep refreshing them in IDAA. Now, of course IDAA will pull the partitions which were updated since last load and what happens also very frequently - the housekeeping kicks in and since the table was heavily updated it qualifies for REORG - on the same partitions of course. This is causing loads to often fail as the tables are often huge (billions of rows) and reorg take time etc.
Now, they all run in shrlevel change. They are not reorg discards and also they do not materialize any changes so in general, Unless it is at the switch phase - I do not see the reason to have those utilities incompatible. We can easily run select * from table during reorg, even giving partition range if we want to as it is shrlevel change. Why would UNLOAD not be able to unload the data from the source dataset and then even apply log changes to it as reorg does? I could see this synched, for example that in case reorg is in switch phase, you want allow to start new unload on the same parts as unloaded. And when unload runs in parallel, switch could just wait for it to finish if it must. Unload is in general fast enough compared with reorg so even standard delay and retry on switch could easily cater for this if there is unload already started.
Why is it useful?
|Who would benefit from this IDEA?||DBAs, application support teams and overall business processes|
How should it work?
We see this to be possible as for other utilities to use standard claims/drains mechanism. By the response from DB2 development in the case:
Over the years, we have made more concurrent utilities compatible
with REORG SHRLEVEL CHANGE, including COPY SHRELVEL CHANGE and LOAD SHRLEVEL CHANGE,
and this would be yet another scenario where it is possible to enable those two
utilities to run parallel, and let the existing Db2 claim/drain processing to
serialize access when REORG needs to drain and to complete the SWITCH phase execution.
The measurable effect of this enhancement will be clearly seen in environments like ours where number of failures and incidents related to this current incompatibility will drop significantly. Also, it will be possble to execute more currently incompatible wrok at the same time and by this, shorten the maintenance window.
|Priority Justification||We clasify this idea as urgent due to hundreds of incidents caused by current design and huge delays in business critical processing times when we cannot serialize reorgs with loads and are not able to process the most current data on the idaa box.|
|Customer Name||Swiss Reinsurance Ltd.|
NOTICE TO EU RESIDENTS: per EU Data Protection Policy, if you wish to remove your personal information from the IBM ideas portal, please login to the ideas portal using your previously registered information then change your email to "firstname.lastname@example.org" and first name to "anonymous" and last name to "anonymous". This will ensure that IBM will not send any emails to you about all idea submissions