Databricks opens its data pipeline troubleshooting tool to public testing
Databricks says it has opened Genie ZeroOps to public testing after a limited trial. The tool can begin an investigation when a data job fails or when a table receives fewer rows than expected. It puts incidents and suggested fixes in a shared inbox for engineers. The company says a person still has to approve changes to production code or data.
Artificial Intelligence··Morning
An alert can open an investigation
Databricks, a US data platform company, says it has opened Genie ZeroOps to beta testers through their workspace administrators. The software watches for conspicuous failures, such as a job that stops with an error, as well as delayed table updates and unexpectedly low row counts. It can start an investigation as soon as it detects a problem and use logs and information on where the data came from to trace the failure.[1]
Engineers can share an incident inbox
The company introduced Genie ZeroOps in June and gave hundreds of customers access before the public beta. The beta adds an inbox that teams can share to see incidents and suggested remedies. Colleagues can tell the software which sources often fail, how they name jobs and which failures they already expect, so that routine interruptions can be filtered out of notifications. An alert can carry a short description of the problem alongside the suggested change.[1]
Production changes still require human approval
For jobs whose code is tracked in Git, engineers can review proposed changes on a separate branch. The software can test a fix away from the live system. If it proposes changes to production code or data, Databricks says those changes still require a person’s approval. Teams can receive incident notifications through Slack, Teams or PagerDuty.[1]
Related columns
For more information on this topic, you can read the related columns.