Prefect Product

Expect the Unexpected

Real-time observability and control with Automations in Prefect Cloud

January 17, 2023
Will Raphaelson
Principal Product Manager
Share

Dataflow coordination is a spectrum. While some tools purely observe or orchestrate, automations in Prefect Cloud bring the best of both worlds to your fingertips—unlocking reactive, stack-aware dataflow at scale. Automations allow you to focus on high-value engineering, knowing that Prefect’s observability system is always monitoring and responding to the world, whatever it throws at you.

Composed of triggers & actions, automations allow users to specify ‘if/then’ logic; if a set of conditions is met, then perform a set of actions. Importantly, and based on feedback from our community members, we know that any reliable workflow system needs to accommodate logic based on the presence or absence of certain conditions. With our design partners and ambassadors in Club42, we’ve implemented a system flexible enough to act on the presence of unexpected events, such as a failing flow run, but also the absence of expected events, such as when an agent stops polling or a run fails to even start. Automations let you move beyond proactive scheduling-based approaches—so that your dataflow can respond automatically and dynamically to observed events in your world.

Today, users can configure automations with triggers that detect in real-time when flows or work queues enter or stay in various states. You can target specific flows, or create broader SLAs for your dataflow based on tags.

Users can configure one or more actions to execute when their trigger criteria are met, including starting or stopping flow runs, work queues, deployment schedules, or sending notifications. Actions are highly parameterizable with Jinja templating, and can even infer their inputs at runtime based on context.

The underlying events architecture on which automations are built is a highly extensible system, capable of ingesting events from across the stack. Prefect is expanding the triggers and actions system to cover more use cases both within and outside of the Prefect universe. Stay tuned for triggers based on block method calls, infrastructure events, and more!

Give automations a try and let us know what you think in the community Slack!