We are running very busy environmenet with IDAA being frequently refreshed by loader with increamnts of inserted records. In addition, we are using the DB2 automated housekeeping approach with the DSNACCOX stored proc implmentation to manage and trigger all required housekeeping/maintenance actions on objects in DB2. There is often a conflict and failure of the load process observed (daily basis, up to circa 100 failures) when the UNLOAD part of idaa load hits the "utility incompatible" error due to reorg being run on the same partition which needs to be unloaded and piped into the IDAA box. etails below.
How it is impacting us - we are running a very busy environment which is highly dependent on the contents of IDAA tables which are frequently loaded. During the critical business periods we frequently insert and update a lot of key tables and at the same time, we need to keep refreshing them in IDAA. Now, of course IDAA will pull the partitions which were updated since last load and what happens also very frequently - the housekeeping kicks in and since the table was heavily updated it qualifies for REORG - on the same partitions of course. This is causing loads to often fail as the tables are often huge (billions of rows) and reorg take time etc.
Now, they all run in shrlevel change. They are not reorg discards and also they do not materialize any changes so in general, Unless it is at the switch phase - I do not see the reason to have those utilities incompatible. We can easily run select * from table during reorg, even giving partition range if we want to as it is shrlevel change. Why would UNLOAD not be able to unload the data from the source dataset and then even apply log changes to it as reorg does? I could see this synched, for example that in case reorg is in switch phase, you want allow to start new unload on the same parts as unloaded. And when unload runs in parallel, switch could just wait for it to finish if it must. Unload is in general fast enough compared with reorg so even standard delay and retry on switch could easily cater for this if there is unload already started.
|Who would benefit from this IDEA?||DBAs, application support teams and overall business processes|
How should it work?
We see this to be possible as for other utilities to use standard claims/drains mechanism. By the response from DB2 development in the case:
Over the years, we have made more concurrent utilities compatible
with REORG SHRLEVEL CHANGE, including COPY SHRELVEL CHANGE and LOAD SHRLEVEL CHANGE,
and this would be yet another scenario where it is possible to enable those two
utilities to run parallel, and let the existing Db2 claim/drain processing to
serialize access when REORG needs to drain and to complete the SWITCH phase execution.
The measurable effect of this enhancement will be clearly seen in environments like ours where number of failures and incidents related to this current incompatibility will drop significantly. Also, it will be possble to execute more currently incompatible wrok at the same time and by this, shorten the maintenance window.
|Priority Justification||We clasify this idea as urgent due to hundreds of incidents caused by current design and huge delays in business critical processing times when we cannot serialize reorgs with loads and are not able to process the most current data on the idaa box.|
|Client Name||Swiss Reinsurance Ltd.|
|IBM's success depends on gathering feedback from customers like yourself. Aha Ideas Portal is the third party tool through which IBM Offering Managers gather feedback from customers such as yourself.|
|IBM is a global organization with business processes, management structures, technical systems and service provider networks that cross borders. As such, the information collected through Aha Ideas Portal (Customer Name, Customer Email Address) will be stored by them in the United States, and handled only as per IBM's instructions and policies. Your data (Name and Email Address) will NOT be shared with other IBM customers.|
|In order to safeguard your information in Aha, do not leave your workstation unattended while using this application, log off after using it, and print only if necessary. If you need to make a hardcopy, remember to pick up the print-out immediately, keep it under lock, and destroy it immediately when no longer needed.|
|NOTICE TO EU RESIDENTS: per EU Data Protection Policy, if you wish to remove your personal information from the IBM ideas portal, please login to the ideas portal using your previously registered information then change your email to "firstname.lastname@example.org" and first name to "anonymous" and last name to "anonymous". This will ensure that IBM will not send any emails to you about all idea submissions|