A source system goes down at 02:00 and comes back at 05:00. By breakfast, one team's dashboards are complete and another team has three empty hours that nobody noticed. Data pipeline automation tools decide that outcome: they schedule pipeline runs, run steps in order, retry failures, rerun past periods and alert someone when a run breaks. Apache Airflow, for one, does not retry a failed task unless someone configures it to [1].
This guide compares 12 data pipeline automation tools on those jobs. It also covers where each one can run, including fully offline, and how each is priced. Figures checked 9 to 13 November 2026.
DataFuseAI publishes this guide and is one of the 12 tools in it. The tools are listed alphabetically, and the Methodology section explains how they were chosen.
Who this is for: data engineers and heads of data choosing how to run recurring pipelines, or replacing cron jobs and scripts. If you are still choosing how to move data between systems, start with what a data integration platform is.
Key Takeaways
- Orchestrators don't retry by default. Apache Airflow's task retries default to 0 [1], and Azure Data Factory's tumbling window retry count defaults to 0 [2]. You configure recovery yourself.
- Managed sync services retry for you. Fivetran keeps retrying a failed sync for up to 14 days before pausing the connection [3]. Airbyte makes up to 5 attempts that sync no data before failing a job [4].
- In our illustrative three-hour outage, hourly interval runs left three hours failed. Cursor-based syncs left none, because the first successful sync picked up every change since the last good one. An orchestrated pipeline that reads from a saved watermark can catch up too, but you have to build it.
- Managed Airflow trails the open-source release. The Airflow docs list 3.4.0 as stable, while Amazon MWAA's newest supported version is 3.3.1 [5][6].
- Four of the 12 document air-gapped or fully offline deployment: Astro Private Cloud, Customer-Managed Prefect, DataFuseAI and Kestra Enterprise [7][8][9][10].
- Price your own workload. Environment-hours, credits, monthly active rows and activity runs measure different things, so list prices don't compare directly.
What does a data pipeline automation tool automate?
A data pipeline automation tool runs a pipeline without a person starting it, and recovers or raises an alarm when a run fails. It covers six jobs: start the run, order the steps, retry failures, rerun past periods, alert a person and keep a run record. Tools differ most on retries and reruns.
We use these six jobs as a checklist throughout this guide. It is a DataFuseAI editorial framework, not an industry standard.
| Job | What it means | The question to ask a vendor |
|---|---|---|
| 1. Trigger | Start a run on a schedule, an event or an upstream change | Can a run start when new data arrives, not only on a clock? |
| 2. Order | Run steps in dependency order | What happens downstream when one step fails? |
| 3. Retry | Try a failed step again, with a delay | Is retry on by default, and how long does it keep trying? |
| 4. Rerun past periods | Backfill or catch up missed hours or days | Does a later success cover the failed period, or does someone rerun it? |
| 5. Alert | Tell a person, in a channel they read | Can alerts wait until the last retry fails? |
| 6. Record | Keep run history and logs | How long is run history kept, and can it be exported? |
Source: DataFuseAI editorial checklist, November 2026.
Jobs 3 and 4 decide how many failures turn into missing data. The outage test later in this guide shows why.
The 12 data pipeline automation tools compared
The 12 tools fall into five groups: open-source orchestrators, managed Airflow services, visual pipeline builders, a data-platform scheduler and managed sync services. Orchestrators run any code you give them, but most start with retries off. Sync services retry on their own, but they are built first to load data.
Figure 1. The 12 tools grouped by what they automate and what you still run. Grouping is DataFuseAI's editorial view; sources are the vendor pages cited in the tables below.
What each tool automates
| Tool | Pipelines defined as | Triggers | Retries out of the box | Rerun past periods | Alerts | |
|---|---|---|---|---|---|---|
| Airbyte | Connections set up in the UI or API | Interval, cron or manual [11] | Yes: up to 5 attempts with no data, 10 with some [4] | Incremental syncs read from the last cursor [12] | Email and webhook in Cloud; webhook only when self-managed [13] | |
Amazon MWAA
| Airflow DAGs in Python | As Airflow, on the version you pick | As Airflow | As Airflow | As Airflow | |
Apache Airflow
| DAGs in Python | Cron, timetables and asset updates [14] | Off: retries default to 0 [1] | Backfill from the UI, CLI or REST API [15] | Callbacks, notifiers and experimental Deadline Alerts [5][16] | |
Astronomer Astro
| Airflow DAGs in Python | As Airflow | As Airflow | As Airflow | As Airflow | |
| Azure Data Factory | Visual pipelines | Schedule, tumbling window, storage event and custom event [17] | Off: tumbling window retry count defaults to 0 [2] | Tumbling window with a past start time [2] | Metric alerts in Monitor [18] | |
| Dagster | Assets and ops in Python | Schedules, sensors and declarative automation [19] | Opt-in; Dagster+ run retries default to 0 [20] | Partition backfills from the UI or CLI [21] | Alert policies, a Dagster+ feature [22] | |
Databricks Lakeflow Jobs
| Jobs made of tasks | Time-based and event-based [23] | Per-task retry policy; serverless jobs adjust retries automatically [24] | Backfill runs over a date range [25] | Email, Slack and webhooks [23] | |
DataFuseAI
| Pipelines on a visual canvas | Cron-based Jobs and manual runs [26] | Not described; fix the cause and re-run [26] | Not described | Notifications switched on per Job [26] | |
| Fivetran | Connections | Fixed interval (default 6 hours), cron on higher plans, or API [3] | Yes: up to 14 days, then the connection pauses [3] | Incremental after the first sync; re-sync to fix integrity errors [3] | Email on new errors, failed syncs and sync delays [27] | |
Google Managed Service for Apache Airflow
| Airflow DAGs in Python | As Airflow | As Airflow | As Airflow | As Airflow | |
| Kestra | Declarative YAML flows, code or no-code editor [28] | Schedules and events [28] | Off: a failed task fails the execution unless retries are set [29] | Backfill from the UI or API; missed schedules recovered by default [30] | Error-handling tasks and alert flows [29] | |
| Prefect | Flows and tasks in Python | Cron and interval schedules, plus event automations [31][32] | Opt-in per task, or as a global default [33] | Runs with past start and end times through the API [34] | Automations send Slack, Teams or email messages [32] | |
Sources: each vendor's documentation, cited in the table, checked 9 to 13 November 2026. "As Airflow" means the managed service runs Apache Airflow, so these jobs work as they do in the Airflow version that service supports.
The table leaves out jobs 2 and 6 because every tool here orders steps and keeps run history in some form. Retention differs by plan, so ask each vendor how long run history is kept and whether it can be exported.
Where each tool runs and how it is priced
| Tool | Where it runs | Fully offline? | Pricing basis, checked 9 Nov 2026 |
|---|---|---|---|
| Airbyte | Cloud; Enterprise Flex data planes in your cloud, VPC or on-premises; free Core on your own infrastructure [35][36] | Core: your design. Flex data planes need outbound access [37]. Self-Managed Enterprise is no longer sold [38] | Standard from $20 a month; Plus from $189 a month; Pro and Enterprise Flex by capacity; Core free under the Elastic License 2.0 [36][39][40][41] |
| Amazon MWAA | AWS; you choose and upgrade the Airflow version [6] | No (AWS service) | Hourly per environment, $0.49 an hour for a small one in AWS's US East example; Serverless bills per task duration [42] |
| Apache Airflow | Wherever you install it | Your design | Free under Apache 2.0; you pay for servers and upkeep [41] |
| Astronomer Astro | Astro cloud; Astro Private Cloud on your own Kubernetes cluster [7] | Yes: Astro Private Cloud supports air-gapped installation [7][43] | Deployments metered hourly, from $0.35 an hour on Developer and $0.42 on Team; workers scale to zero when idle; higher tiers quoted [44] |
| Azure Data Factory | Azure; a self-hosted integration runtime runs activities in hybrid networks [45] | No (Azure service) | Per 1,000 orchestration runs plus integration runtime hours, with separate self-hosted rates; check the Azure calculator for your region [45] |
| Dagster | Dagster+ Serverless or Hybrid (ECS, Kubernetes, Docker, Azure, local); open source anywhere [46] | Open source: your design | Solo $10 a month plus $0.040 a credit; Starter $100 plus $0.035; Hybrid has no compute charge [47] |
| Databricks Lakeflow Jobs | Your Databricks workspace | No (Databricks service) | Databricks Units (DBUs); serverless billed per second [48] |
| DataFuseAI | Managed, private-hosted or local on-premise [9] | Yes, with no internet dependency [9] | Not published; a 30-day POC is offered [49] |
| Fivetran | SaaS; Hybrid agent in your environment; HVR, its self-hosted version [50][51] | HVR runs on your own servers; offline use not stated in the pages cited | Monthly active rows per connection; $5 base charge from 1 to 1 million MAR [52] |
| Google Managed Service for Apache Airflow | Google Cloud | No (Google Cloud service) | DCU-hours, $0.06 per DCU-hour in us-central1 [53] |
| Kestra | Open source on Docker, Kubernetes or a VM; Enterprise in your infrastructure; Kestra Cloud on Google Cloud [10] | Yes, Enterprise Edition [10] | Open source free under Apache 2.0; Enterprise per instance, annual, quoted; Cloud usage-based [41][54] |
| Prefect | Prefect Cloud with hybrid work pools; Customer-Managed Prefect in your infrastructure; open-source server anywhere [8][55] | Yes: Customer-Managed Prefect runs fully air-gapped [8] | Hobby free; Starter $100 a month; Team $100 per user a month; Enterprise custom [55] |
Sources: vendor pricing and deployment pages and GitHub LICENSE files, cited in the table, checked 9 November 2026; Airbyte and Astronomer prices and the offline column re-checked 13 November 2026.
Prices are list prices from each vendor's page, in US dollars, before tax, and they cover different things. Fivetran's MAR is not affected by sync frequency, because it counts unique rows changed, not how often they sync [52]. An MWAA environment bills by the hour whether or not DAGs run [42]. Astro bills deployments hourly but scales workers to zero when idle [44].
Which data pipeline automation tool should you choose?
Choose by what your pipelines already are. If they are Python, pick an orchestrator and decide who runs it. If you only need to land SaaS and database data in a warehouse, a managed sync service retries for you. If you live in one cloud or platform, its native service is the shortest path.
| Your situation | Start with | Why, from the tables above |
|---|---|---|
| Existing Airflow DAGs, no wish to run servers | Amazon MWAA, Astronomer Astro or Google Managed Service for Apache Airflow | Same DAG code; the provider runs the scheduler. Check which Airflow version each supports |
| New Python pipelines built around tables and files | Dagster or Prefect | Both are Apache 2.0 and offer a hybrid cloud that runs your code on your infrastructure |
| Pipelines written as declarative YAML | Kestra | Schedules, events, retries and backfills are documented for YAML flows |
| An air-gapped or fully offline site | Astro Private Cloud, Customer-Managed Prefect, DataFuseAI or Kestra Enterprise | All four document air-gapped or no-internet deployment; choose by Airflow, Python flows, visual pipelines or YAML |
| Loading SaaS and databases into a warehouse | Fivetran or Airbyte | Retries and incremental catch-up are built in; check what each offers for steps after the load |
| Everything already in Azure or Databricks | Azure Data Factory or Databricks Lakeflow Jobs | Native triggers, monitoring and billing in the platform you already pay for |
| Visual pipelines that must run inside your own network | DataFuseAI, or Azure Data Factory with a self-hosted runtime | DataFuseAI runs offline but describes no automatic retries; ADF documents retries and backfills but is managed in Azure |
Source: DataFuseAI editorial guidance derived from the documented facts in the two tables above, checked 9 November 2026. It is not a ranking.
What happens after a failure? An illustrative outage test
We modelled one hourly pipeline whose source is unavailable from 02:00 to 05:00, using each tool's documented default retry rules. Fivetran and Airbyte recovered all three hours without a person. Airflow and Azure Data Factory, on their defaults and with each run processing only its own hour, left three hours failed. Adding three Airflow retries five minutes apart did not change that.
Figure 2. Illustrative model, not a benchmark. Each row applies one tool's documented default or one stated setting; the rules are listed in the Methodology section.
| Behaviour modelled (illustrative) | Rule used | Failed attempts | Hours left failed |
|---|---|---|---|
| Fivetran, hourly sync | Retry 20 minutes after each failure, early in an outage [3] | 9 | 0 |
| Airbyte, hourly schedule | Five quick attempts per job (Airbyte Core defaults, configurable), then the next scheduled sync [4][11] | 15 | 0 |
| Apache Airflow, defaults | No retries; each hourly run processes only its own hour [1] | 3 | 3 |
| Apache Airflow, retries=3 | Three retries, 5 minutes apart | 12 | 3 |
| Azure Data Factory tumbling window, defaults | Retry count 0; one run per window [2] | 3 | 3 |
Source: DataFuseAI illustrative model (Python) using the documented retry rules cited in the table, checked 9 November 2026.
The difference comes from how each pipeline reads data. Fivetran and Airbyte sync incrementally from a saved cursor, so the 05:00 sync also picks up changes made during the outage [3][12]. An interval run in Airflow or a tumbling window in Azure Data Factory processes only its own hour. When that hour fails, it stays failed until someone clears it or backfills it [1][2]. These rows assume each run processes only its own hour, a common design. An Airflow or Azure Data Factory pipeline that reads from a saved watermark would catch up the same way, but you have to build that logic.
Retries only cover outages shorter than the retry window. Three retries five minutes apart cover about 15 minutes. Azure Data Factory's shortest retry interval is 30 seconds [2]. For longer outages, plan a backfill or catch-up step as part of the pipeline, not as an incident task.
The cursor approach has its own limit. Airbyte notes that if records change without their cursor field being updated, incremental syncs miss those changes [56].
The 12 tools in brief
Airbyte
Airbyte syncs sources to destinations on a schedule, with automatic attempts and backoff [4]. Enterprise Flex keeps the data plane in your environment while Airbyte runs the control plane [35]. Watch for: the core code is under the Elastic License 2.0, not an OSI open-source licence [41].
Amazon MWAA
Amazon MWAA runs Apache Airflow inside AWS, but you choose the Airflow version and upgrades are your job [6]. Its newest supported version on 9 November 2026 was 3.3.1. Watch for: an environment bills by the hour whether or not DAGs run [42].
Apache Airflow
Apache Airflow is the open-source orchestrator that three other services on this list run for you. Version 3 renamed datasets to assets and extended asset-based scheduling, and Deadline Alerts arrived in 3.1 as an experimental feature [14][16]. Watch for: retries are off by default, and you run the scheduler, database and workers yourself.
Astronomer Astro
Astronomer Astro is managed Airflow, billed by deployment and worker hours [44]. Astro Private Cloud runs it on your own Kubernetes cluster, including air-gapped sites [7][43]. Watch for: deployments bill hourly by size even when workers have scaled to zero [44].
Azure Data Factory
Azure Data Factory is Microsoft's pipeline service, with schedule, tumbling window and event triggers [17]. A self-hosted integration runtime lets activities reach on-premises systems [45]. Watch for: Microsoft now calls Data Factory in Microsoft Fabric "the next generation" of ADF [2], so check the product's direction before a large build.
Dagster
Dagster models pipelines as data assets, with partitions, backfills and asset-level automation [19][21]. Dagster+ bills a base fee plus credits, where one credit is an asset materialization or op execution [46]. Watch for: Prefect announced it is acquiring Dagster Labs, and Dagster says customer pricing and contracts are unchanged [57][58].
Databricks Lakeflow Jobs
Databricks Lakeflow Jobs orchestrates data processing tasks inside Databricks, with time and event triggers, retries and notifications [23][24]. Backfill runs became generally available in October 2025 [59]. Watch for: it is built for work inside your Databricks workspace, and backfills need date parameters your job reads [25].
DataFuseAI
DataFuseAI builds pipelines on a visual canvas and runs them as cron-based Jobs, with run history and step logs [26]. It can run managed, private-hosted or fully offline on your own network [9]. Watch for: automatic retries and backfills are not described on its automation page.
Fivetran
Fivetran runs managed connector syncs and keeps retrying failures on a documented schedule [3]. HVR is its self-hosted version [51]. Watch for: billing is per connection, with a $5 base charge on low-volume connections [52].
Google Managed Service for Apache Airflow
Google's managed Airflow appears on its pricing page as Managed Service for Apache Airflow (Gen 3), while its billing SKUs still use the Cloud Composer 3 name [53]. Gen 3 bills by DCU-hours, a measure of provisioned resources. Watch for: search for both names in Google's documentation.
Kestra
Kestra defines pipelines as declarative YAML flows and documents event triggers, retries and backfills [29][30]. Enterprise Edition adds governance and can run fully air-gapped [10]. Watch for: SSO, RBAC and audit logs are not in the open-source edition [10].
Prefect
Prefect runs Python flows, with the control plane in Prefect Cloud and workers on your infrastructure through work pools [55]. Retries are set per task or as a global default [33]. Watch for: hybrid execution is listed from the Starter plan up, and the Dagster acquisition may change both product lines over time [55][57].
Can you run pipeline automation fully on-premises?
Yes. Four of the 12 document air-gapped or fully offline deployment: Astro Private Cloud, Customer-Managed Prefect, DataFuseAI and Kestra Enterprise. The Apache 2.0 orchestrators run wherever you install them. Hybrid models keep data local but still call a vendor control plane.
- Air-gapped or offline, documented: Astro Private Cloud supports air-gapped installs with private container registries [43]. Prefect says Customer-Managed Prefect runs fully air-gapped, with control plane and workers in your network [8]. Kestra Enterprise runs workers and, where required, the control plane inside the air-gapped site [10]. DataFuseAI's local model has no internet dependency [9].
- Self-hosted, offline depends on your setup: Apache Airflow, Dagster and open-source Prefect are Apache 2.0 [41], and a self-hosted Prefect server needs no Prefect-hosted endpoints for core operation [60]. Airbyte Core and Fivetran's HVR also run on your own servers [36][51]. Airbyte no longer sells its Self-Managed Enterprise edition [38].
- Hybrid, needs outbound access: Airbyte Enterprise Flex data planes need egress to Airbyte's control plane [37], and Fivetran's Hybrid agent communicates outbound with Fivetran [61].
Ask every vendor what leaves your network: run metadata, logs, credentials or none. Our on-premise ETL guide separates self-hosted from fully offline in more detail.
Mistakes to avoid
- Assuming retries are on. Airflow, Dagster+ run retries and Azure Data Factory tumbling windows all default to zero [1][2][20].
- Treating retries as recovery. Retries cover short blips. A failed interval still needs a backfill or a catch-up design.
- Alerting on every attempt. Alert after the last retry fails, or the team learns to ignore the channel. Lakeflow Jobs can mute notifications until the last retry, and Dagster+ asset alerts wait for retries to finish [22][62].
- Comparing prices on different units. Environment-hours, credits, MAR and activity runs measure different things. Price your own workload on each.
- Watching run status only. A green run can still load wrong data. Our post on data observability covers what status checks miss.
- Ignoring version lag. Managed Airflow trails the open-source release, and on MWAA, upgrades are your responsibility [6].
Where DataFuseAI fits (and where it doesn't)
DataFuseAI fits teams that want to build pipelines visually and run them on a schedule inside their own network, including fully offline [9]. Its pipeline automation saves a pipeline as a Job with a cron recurrence, runs several pipelines in a set order and keeps step-level logs [26].
It is a weaker fit if you need automatic retries, partition backfills or event-driven triggers, because its automation page describes none of them [26]. It is built around a visual canvas rather than Python code, so teams with large Airflow or Dagster codebases should keep them.
To test it on one of your own workflows, DataFuseAI offers a 30-day proof of concept for one recurring workflow and two or three sources, with no POC fee [49]. Applications are reviewed, and any external cloud cost is yours. Our ETL proof-of-concept framework explains what to test first.
Frequently Asked Questions
What is the difference between orchestration and data pipeline automation?
Orchestration is one part of data pipeline automation: deciding when tasks run and in what order. Pipeline automation also covers retries, reruns of past periods, alerts and run history. Orchestrators like Apache Airflow handle ordering for any code. Managed sync services like Fivetran automate scheduling and retries for data movement only, so many teams use both.
Is Apache Airflow still a good choice in 2027?
Apache Airflow is still actively released, with 3.4.0 listed as the stable documentation version on 9 November 2026. Three managed services on this list run it. Its weak points are operating it yourself and retries that start switched off. Teams starting fresh can also compare Dagster, Prefect and Kestra.
Does Prefect's acquisition of Dagster change which one to pick?
Not yet. Prefect announced on 13 July 2026 that it is acquiring Dagster Labs. Dagster says it continues under its own brand, and that pricing, contracts and support are unchanged [57][58]. Both open-source projects stay Apache 2.0 today. If you choose either, ask about the roadmap in writing and keep pipeline code portable.
Why isn't dbt on this list?
dbt transforms data inside a warehouse rather than scheduling whole pipelines, so we treated it as a step that pipeline automation tools run. Fivetran and dbt Labs completed their merger on 1 June 2026 [63]. Airflow, Dagster, Prefect and Kestra can all run dbt as one step in a larger pipeline with retries and alerts.
Can I use a sync service and an orchestrator together?
Yes, and it is a common pattern. A sync service such as Fivetran or Airbyte loads raw data and retries its own failures. An orchestrator then runs the transformation, checks and exports in order. Make the orchestrator wait for the sync to finish, and send one alert path, so a single failure doesn't raise two separate alarms.
Methodology
Selection. We included tools whose documented main job is scheduling and running data pipelines, or running managed data syncs on a schedule, and that publish documentation on triggers, retries and alerts. The three managed Airflow services are listed separately because who runs and upgrades Airflow differs. Matillion, Informatica, Qlik Talend, Hevo, AWS Glue, AWS Step Functions, Mage and Temporal were left out for space, not for anything we found. DataFuseAI does not publish retry or backfill documentation, so it does not meet this rule. It is included as an exception because DataFuseAI publishes this guide.
Sources and dates. Every feature and price comes from the vendor's own documentation or pricing page, read on 9 November 2026 (UTC). Airbyte and Astronomer prices, Azure Data Factory's pricing page, the offline column and the Prefect–Dagster deal status were re-checked on 13 November 2026. Open-source licences were read from each project's LICENSE file on GitHub. Prices are list prices in US dollars before tax and discounts. Where a page does not state something, the tables say "not stated" or "not described".
Outage model. The outage test is an illustrative Python model, not a measurement of any product. It assumes an hourly pipeline, a source that is unavailable from 02:00 to 05:00, and each tool's documented retry rule. Fivetran retries 20 minutes after a failure for a one-hour sync frequency. Airbyte makes up to five attempts per job with backoff of 10, 30, 90 and 270 seconds, which are Airbyte Core's configurable defaults. Airflow and Azure Data Factory run one interval per hour with no retries, plus one Airflow run with three retries five minutes apart. It does not model connector-specific behaviour, rate limits, partial loads or source data retention.
References
- Astronomer. "Rerun Airflow Dags and tasks." Accessed 9 November 2026. https://www.astronomer.io/docs/airflow/rerunning-dags
- Microsoft Learn. "Create a trigger that runs a pipeline on a tumbling window." Accessed 9 November 2026. https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-tumbling-window-trigger
- Fivetran. "Sync Overview." Accessed 9 November 2026. https://fivetran.com/docs/getting-started/syncoverview
- Airbyte. "Workloads & Jobs." Accessed 9 November 2026. https://docs.airbyte.com/platform/understanding-airbyte/jobs
- Apache Software Foundation. "Callbacks." Airflow 3.4.0 documentation. Accessed 9 November 2026. https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/logging-monitoring/callbacks.html
- Amazon Web Services. "Apache Airflow versions on Amazon Managed Workflows for Apache Airflow." Accessed 9 November 2026. https://docs.aws.amazon.com/mwaa/latest/userguide/airflow-versions.html
- Astronomer. "Astro Private Cloud features." Accessed 13 November 2026. https://www.astronomer.io/docs/astro-private-cloud/apc-features
- Prefect. "Prefect Customer Managed" and "Welcome to Customer-Managed Prefect." Accessed 13 November 2026. https://www.prefect.io/prefect/customer-managed and https://docs-customer-managed.prefect.io/
- DataFuseAI. "Deployment Options." Accessed 9 November 2026. https://www.datafuseai.com/platform/deployment-options
- Kestra. "Security." Accessed 9 November 2026. https://kestra.io/security
- Airbyte. "Sync schedules." Accessed 9 November 2026. https://docs.airbyte.com/using-airbyte/core-concepts/sync-schedules
- Airbyte. "Sync Modes." Accessed 9 November 2026. https://docs.airbyte.com/using-airbyte/core-concepts/sync-modes/
- Airbyte. "Manage notifications." Accessed 9 November 2026. https://docs.airbyte.com/platform/cloud/managing-airbyte-cloud/manage-airbyte-cloud-notifications
- Apache Software Foundation. "Release Notes: Airflow 3.0.0." Accessed 9 November 2026. https://airflow.apache.org/docs/apache-airflow/3.0.0/release_notes.html
- Apache Software Foundation. "Backfill." Airflow 3.1.2 documentation. Accessed 9 November 2026. https://airflow.apache.org/docs/apache-airflow/3.1.2/core-concepts/backfill.html
- Apache Software Foundation. "Deadline Alerts." Airflow 3.3.2 documentation. Accessed 9 November 2026. https://airflow.apache.org/docs/apache-airflow/3.3.2/howto/deadline-alerts.html
- Microsoft Learn. "Pipeline execution and triggers in Azure Data Factory or Azure Synapse Analytics." Accessed 9 November 2026. https://learn.microsoft.com/azure/data-factory/concepts-pipeline-execution-triggers
- Microsoft Learn. "Visually monitor Azure Data Factory." Accessed 9 November 2026. https://learn.microsoft.com/azure/data-factory/monitor-visually
- Dagster Labs. "Automate" (schedules, sensors and declarative automation). Accessed 9 November 2026. https://docs.dagster.io/guides/automate
- Dagster Labs. "Run retries" and "Deployment settings reference." Accessed 9 November 2026. https://docs.dagster.io/deployment/run-retries and https://docs.dagster.io/dagster-cloud/developing-testing/deployment-settings-reference
- Dagster Labs. "Backfilling data." Accessed 9 November 2026. https://docs.dagster.io/overview/partitions-backfill/backfill
- Dagster Labs. "Alert policy types." Accessed 9 November 2026. https://docs.dagster.io/dagster-plus/features/alerts/alert-policy-types
- Databricks. "Lakeflow Jobs." Accessed 9 November 2026. https://docs.databricks.com/jobs
- Databricks. "Configure and edit tasks in Lakeflow Jobs." Accessed 9 November 2026. https://docs.databricks.com/jobs/configure-task
- Databricks. "Backfill jobs." Accessed 9 November 2026. https://docs.databricks.com/en/jobs/backfill-jobs
- DataFuseAI. "Data Pipeline Automation, Scheduling & Monitoring." Accessed 9 November 2026. https://www.datafuseai.com/solutions/data-pipeline-automation
- Fivetran. "Notifications." Accessed 13 November 2026. https://fivetran.com/docs/using-fivetran/fivetran-dashboard/user-management/notifications
- Kestra. "Enterprise." Accessed 9 November 2026. https://kestra.io/enterprise
- Kestra. "Handle Errors in Kestra: Retries and Alerts." Accessed 9 November 2026. https://kestra.io/docs/tutorial/errors
- Kestra. "Run Flows on a Cron Schedule with Backfills and Conditions." Accessed 9 November 2026. https://kestra.io/docs/workflow-components/triggers/schedule-trigger
- Prefect. "Schedule flow runs." Accessed 9 November 2026. https://docs.prefect.io/v3/concepts/schedules
- Prefect. "Automations." Accessed 9 November 2026. https://docs.prefect.io/v3/concepts/automations
- Prefect. "Retries." Accessed 9 November 2026. https://docs.prefect.io/v3/how-to-guides/workflows/retries
- Prefect. "Schedule Deployment" (REST API reference). Accessed 9 November 2026. https://docs.prefect.io/v3/api-ref/rest-api/server/deployments/schedule-deployment
- Airbyte. "Hybrid deployment" (Enterprise Flex). Accessed 9 November 2026. https://docs.airbyte.com/platform/hybrid-deployment
- Airbyte. "Airbyte Cloud" (plans and pricing). Accessed 9 November 2026. https://airbyte.com/solutions/airbyte-cloud
- Airbyte. "Deploy a data plane with Airbox in Enterprise Flex." Accessed 9 November 2026. https://docs.airbyte.com/platform/enterprise-flex/data-plane-util
- Airbyte. "Airbyte Self-Managed Enterprise." Accessed 13 November 2026. https://docs.airbyte.com/enterprise-setup/
- Airbyte. "Airbyte Pricing." Accessed 13 November 2026. https://airbyte.com/pricing
- Airbyte. "Airbyte Enterprise" (pricing calculator, Plus plan). Accessed 13 November 2026. https://airbyte.com/solutions/airbyte-enterprise
- GitHub. LICENSE files of apache/airflow, dagster-io/dagster, PrefectHQ/prefect, kestra-io/kestra and airbytehq/airbyte. Accessed 9 November 2026. https://github.com/apache/airflow https://github.com/dagster-io/dagster https://github.com/PrefectHQ/prefect https://github.com/kestra-io/kestra https://github.com/airbytehq/airbyte
- Amazon Web Services. "Amazon MWAA pricing." Accessed 9 November 2026. https://aws.amazon.com/managed-workflows-for-apache-airflow/pricing/
- Astronomer. "Astro Private Cloud." Accessed 13 November 2026. https://www.astronomer.io/product/astro-private-cloud
- Astronomer. "Astro pricing." Accessed 9 and 13 November 2026. https://www.astronomer.io/pricing
- Microsoft Azure. "Data Pipeline pricing." Accessed 13 November 2026. https://azure.microsoft.com/en-us/pricing/details/data-factory/v2/
- Dagster Labs. "Pricing." Accessed 9 November 2026. https://dagster.io/pricing
- Dagster Labs. "Dagster+ Solo and Starter pricing updates (May 2026)." Accessed 9 November 2026. https://support.dagster.io/articles/3171123463-dagster-solo-and-starter-pricing-updates-may-2026
- Databricks. "How does serverless pricing work?" Accessed 9 November 2026. https://answers.databricks.com/does-serverless-pricing-work
- DataFuseAI. "Free 30-Day POC." Accessed 9 November 2026. https://www.datafuseai.com/free-poc
- Fivetran. "Deployment Models." Accessed 9 November 2026. https://fivetran.com/docs/deployment-models
- Fivetran. "HVR 6 Documentation." Accessed 13 November 2026. https://fivetran.com/docs/hvr6
- Fivetran. "Usage-Based Pricing" and "Monitor and Optimize Usage." Accessed 9 November 2026. https://www.fivetran.com/docs/usage-based-pricing and https://fivetran.com/docs/usage-based-pricing/tracking-and-optimizing-usage
- Google Cloud. "Managed Service for Apache Airflow pricing." Accessed 9 November 2026. https://cloud.google.com/composer/pricing
- Kestra. "Pricing." Accessed 9 November 2026. https://www.kestra.io/pricing
- Prefect. "Prefect Cloud pricing." Accessed 9 November 2026. https://www.prefect.io/pricing
- Airbyte. "Incremental Sync - Append." Accessed 9 November 2026. https://docs.airbyte.com/platform/using-airbyte/core-concepts/sync-modes/incremental-append
- Dagster Labs. "Prefect is acquiring Dagster." 13 July 2026. https://dagster.io/blog/prefect-is-acquiring-dagster
- Dagster Labs. "Dagster Customer FAQ: Prefect Acquisition." Accessed 9 November 2026. https://dagster.io/dagster-customer-faq
- Databricks. "Announcing Backfill Runs in Lakeflow Jobs for Higher Quality Downstream Data." 21 October 2025. https://www.databricks.com/blog/announcing-backfill-runs-lakeflow-jobs-higher-quality-downstream-data
- Prefect. "Configure network access." Accessed 13 November 2026. https://docs.prefect.io/v3/advanced/configure-network-access
- Fivetran. "Hybrid Deployment." Accessed 9 November 2026. https://fivetran.com/docs/core-concepts/deployment-models/hybrid-deployment
- Microsoft Learn. "Configure automatic restarts" (Implement Lakeflow Jobs training module). Accessed 9 November 2026. https://learn.microsoft.com/en-us/training/modules/implement-lakeflow-jobs/6-configure-job-automatic-restarts
- Fivetran (Business Wire). "Fivetran + dbt Labs Complete Merger to Create the Data Infrastructure for Trusted AI Agents." 1 June 2026. https://www.businesswire.com/news/home/20260601514374/en/Fivetran-dbt-Labs-Complete