IBM Data and AI

Welcome to the IBM Data and AI Ideas Portal for Clients!

We welcome and appreciate your feedback on IBM Data and AI Products to help make them even better than they are today!
Before you submit an idea, please perform a search first as a similar idea may have already been reported in the portal. If a related idea is not yet listed, please create a new idea and include with it a description which includes expected behavior as well as why having this feature would improve the service and how it would address your use case.
IBM Employees:
Clients:
  • Our team welcomes any feedback and suggestions you have for improving our offerings / products! This forum allows us to connect your offering / product improvement ideas with IBM product and engineering teams.

  • If you have not registered on this portal please click on the following link and register. To complete registration you will need to open the email you will receive from Aha to confirm your identity. http://ibm.biz/IBM-Data-and-AI-Portal-Register

Additional Information:
  • The shorter URL for this site is: https://ibm.biz/IBM-Data-and-AI-Ideas

  • To view our roadmaps: http://ibm.biz/Data-and-AI-Roadmaps

  • Reminder: This is not the place to submit defects or support needs, please use normal support channel for these cases

  • Please do not use the Ideas Portal for reporting bugs - we ask that you report bugs or issues with the product by contacting IBM support.

DataStage DFD REST API should support stopping a running job and resetting a failed job

We are using DataStage to move large volumes of data into our data lake. We have our own UI our users use to run jobs that ingest their data into our data lake. The reason we have our UI instead having our users use DFD directly is to meet governance requirements. Our UI calls DataStage DFD REST API to run DataStage jobs that move data but as there is no API for stopping a running job we are not able to meet our requirement of giving users the ability to stop a job that they had started.

The DataStage sequence jobs we run via the DFD REST API has the 'Allow multiple instances' option enabled so we can run the same job concurrently with different parameters and invocation ID. We had to design these jobs so that they never abort because DFD REST API does not support resetting a job if it were to ever abort, unlike the DataStage Director client which lets users reset a job or a particular invocation ID of a job. We report back to the user the completion status of these top level sequence jobs exactly as DataStage reports it because we do not want to maintain the complexity of having to check the completion status of all Job Activity stages run by the sequence to see if one of them aborted and if so, report the entire sequence as failed to the user. As a result currently we always report to users that the job succeeds and they'd have to look in the logs to find that actually one of the activities in the sequence failed and so the data movement actually failed. If DFD REST API supports resetting a job invocation it will enable us to let sequence jobs abort when they actually should abort and report completion status accurately.

  • Avatar32.5fb70cce7410889e661286fd7f1897de Guest
  • Jul 24 2020
  • Future consideration
Idea Priority High
Customer Name IBM Chief Data Office Cognitive Enterprise Data Platform
  • Attach files

NOTICE TO EU RESIDENTS: per EU Data Protection Policy, if you wish to remove your personal information from the IBM ideas portal, please login to the ideas portal using your previously registered information then change your email to "anonymous@euprivacy.out" and first name to "anonymous" and last name to "anonymous". This will ensure that IBM will not send any emails to you about all idea submissions