IBM Data and AI

Welcome to the IBM Data and AI Ideas Portal for Clients!

We welcome and appreciate your feedback on IBM Data and AI Products to help make them even better than they are today!
Before you submit an idea, please perform a search first as a similar idea may have already been reported in the portal. If a related idea is not yet listed, please create a new idea and include with it a description which includes expected behavior as well as why having this feature would improve the service and how it would address your use case.
IBM Employees:
Clients:
  • Our team welcomes any feedback and suggestions you have for improving our offerings / products! This forum allows us to connect your offering / product improvement ideas with IBM product and engineering teams.

  • If you have not registered on this portal please click on the following link and register. To complete registration you will need to open the email you will receive from Aha to confirm your identity. http://ibm.biz/IBM-Data-and-AI-Portal-Register

Additional Information:
  • The shorter URL for this site is: https://ibm.biz/IBM-Data-and-AI-Ideas

  • To view our roadmaps: http://ibm.biz/Data-and-AI-Roadmaps

  • Reminder: This is not the place to submit defects or support needs, please use normal support channel for these cases

  • Please do not use the Ideas Portal for reporting bugs - we ask that you report bugs or issues with the product by contacting IBM support.

Q Replication to Kafka Targets

Kafka evolves as a platform for real time data streaming, analytics, and event processing. For enterprise customers the challenge is to let data originating on IBM z flow to Kafka. Popular options are Kafka APIs, MQ, data replication. Only data replication offers mechanisms to let data flow to Kafka without to change existing applications.

IBM supports data replication to Kafka with CDC technology. We have a significant Q Replication footprint already, running Q Replication on dozens of systems for thousands of tables. The effort of introducing another data replication technology with the desired degree of monitoring and automation just because Q Replication does not support Kafka targets would be significant and should be avoided. We consider that the Q Rep operational model would fit very well to support Kafka targets.

  • Avatar32.5fb70cce7410889e661286fd7f1897de Guest
  • May 11 2020
  • Not Under Consideration
Why is it useful?
Who would benefit from this IDEA? All enterprise customers with existing Q Replication footprint. The more systems and the more tables replicated with Q Rep today, the bigger the benefit.
How should it work?

The desired solution should be able to read Q Rep's compact message format, convert the data to unicode, and feed a Kafka producer to store the replicated data as Kafka messages. One topic per replicated table.

Especially for existing Q Rep enterprise customers the benefit would be high.

Idea Priority Medium
Priority Justification Established procedures could be used to manage the data replication to Kafka, MIPS could be saved significantly, because no additional log reader has to be run on the source platform, admin manpower would be saved to set up and monitor the solution.
Customer Name Katrin Weiher
  • Attach files
  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    21 May 17:59

    Most of our publications are headed to KAFKA anyway, so this would save a lot of work. We have Qrep installed, but do not have CDC implemented. Would prefer that Qrep get this new feature.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    12 May 13:12

    This should be put higer priority since popular demands from China customers.

NOTICE TO EU RESIDENTS: per EU Data Protection Policy, if you wish to remove your personal information from the IBM ideas portal, please login to the ideas portal using your previously registered information then change your email to "anonymous@euprivacy.out" and first name to "anonymous" and last name to "anonymous". This will ensure that IBM will not send any emails to you about all idea submissions