IBM Data & AI

 Welcome to the IBM Data & AI Ideas Portal for Clients! 

We welcome and appreciate your feedback on IBM Data & AI Products to help make them even better than they are today!
Before you submit an idea, please perform a search first as a similar idea may have already been reported in the portal.  If a related idea is not yet listed, please create a new idea and include with it a description which includes expected behavior as well as why having this feature would improve the service and how it would address your use case.
IBM Employees:
Clients:
  • Our team welcomes any feedback  and suggestions you have for improving our offerings / products!  This forum allows us to connect your offering / product improvement ideas with IBM product and engineering teams.
  • If you have not registered on this portal please click on the following link and register.  To complete registration you will need to open the email you will receive from Aha to confirm your identity. http://ibm.biz/IBM-Data-and-AI-Portal-Register
Additional Information:
  • The shorter URL for this site is: https://ibm.biz/IBM-Data-and-AI-Ideas
  • To view our roadmaps: http://ibm.biz/Data-and-AI-Roadmaps
  • Reminder: This is not the place to submit defects or support needs, please use normal support channel for these cases
  • Please do not use the Ideas Portal for reporting bugs - we ask that you report bugs or issues with the product by contacting IBM support.

kafka connector should be able to read only committed messages

Our Kafka Producer writes messages on a Kafka topic acting transactionally and it could rollback the transaction when certain events occur. As of now, the behaviour of the Kafka connector is to read also uncommitted messages (see case TS003289786), meaning that rolled back messages will be ingested. The Kafka Connector should be able to let the user decide if reading only committed messages (isolation.level=READ_COMMITTED) or also non committed messages (READ_UNCOMMITTED)

  • Guest
  • Feb 3 2020
  • Planned
Why is it useful?
Who would benefit from this IDEA? As a customer, we need to have a transactional behaviour of the kafka consumer in order to use Big Integrate and Information Server to interact with our Kafka Cluster
How should it work?

1) The Kafka Connector lets the user define the behaviour of the KafkaConsumer class

2) The isolation.level property will be set accordingly and the transactional consumer should be developed as per the Kafka guidelines (https://cwiki.apache.org/confluence/display/KAFKA/Transactional+Messaging+in+Kafka
)

Idea Priority Urgent
Priority Justification It is a mandatory requirement for the italian ministry of finance. The penalty would be the choice of a different system to ingest data
Customer Name SOGEI
Submitting Organization
Submitter Tags
  • Attach files

NOTICE TO EU RESIDENTS: per EU Data Protection Policy, if you wish to remove your personal information from the IBM ideas portal, please login to the ideas portal using your previously registered information then change your email to "anonymous@euprivacy.out" and first name to "anonymous" and last name to "anonymous". This will ensure that IBM will not send any emails to you about all idea submissions