APACHE AIRFLOW · INDEPENDENT ENGINEERING
Design, upgrade and operate self-managed Apache Airflow, from executors and Dag standards to the move from Airflow 2 to Airflow 3, tested against your own pipelines.
Independent Apache Airflow engineering.
These are independent Bassirah engineering and support offerings. They do not imply upstream vendor or foundation endorsement, official vendor support rights, enterprise feature entitlement or maintenance guarantees.
Apache Airflow, Airflow and Apache are either registered trademarks or trademarks of The Apache Software Foundation. Bassirah is not a partner of Astronomer, Amazon Web Services or Google Cloud.
Supported releases.
Apache Airflow is a project of The Apache Software Foundation, developed by its community and released under the Apache License 2.0. Bassirah provides independent engineering for the Airflow 3 release line, with 3.3 as the default target, and plans upgrades for deployments still on Airflow 2. The exact versions in scope are named in each statement of work. Last reviewed: 26 October 2026.
| Release line | Status on 26 October 2026 | How Bassirah treats it |
|---|---|---|
| Airflow 3.3 | Current feature release: 3.3.0 on 6 July 2026; latest patch 3.3.2, 17 September 2026. Airflow 3 is in maintenance; end-of-life date not yet set by the project | Default target for new deployments and upgrades |
| Airflow 3.4 | Pre-release only: 3.4.0 beta 2, 20 October 2026 | Tested in non-production; not a production target until a final release |
| Airflow 3.0 to 3.2 | Earlier feature releases of Airflow 3; the last patches were 3.0.6, 3.1.8 and 3.2.2 | Upgrade to the latest 3.3 patch release |
| Airflow 2 (last release 2.11.2) | End of life since 22 April 2026, after limited maintenance from 22 October 2025 | Upgrade assessment and planned move to Airflow 3 |
WHY IT MATTERS
Apache Airflow is a community platform for authoring, scheduling and monitoring workflows in Python. Teams define pipelines as Dags, and Airflow runs each task in order, on schedule or when upstream data changes.
Airflow 3 changed the architecture. Tasks now report state through the API server instead of writing to the metadata database, a separate Dag processor parses pipeline code, and Dag versioning, assets and event-driven scheduling are built in. Many teams still carry Airflow 2 Dags, plugins and operators that depend on removed features. Airflow also runs the code your Dag authors write, so executor choice and isolation are security decisions as well as capacity decisions. Bassirah tests these changes against your own Dags and engineers the deployment so it holds up in production.
WHAT BASSIRAH DELIVERS
FIT FIRST
Airflow adds services to run, secure and upgrade. We recommend running it yourself when the workloads need what it provides, and we say so when a simpler scheduler or a managed service is enough.
| Requirement | Self-managed Airflow is worth evaluating when… | Something simpler is often enough when… |
|---|---|---|
| Dependencies | Pipelines span many systems, with dependencies, retries and backfills that need one place to define and observe them | A few independent jobs run on fixed schedules, and cron or the tool's own scheduler covers them |
| Code and review | Teams want pipelines in Python, under version control, with reviews and automated tests | Pipelines are built in a low-code tool that already schedules them |
| Control | You need to choose the executor, network placement, isolation and upgrade timing yourself | A managed Airflow service from your cloud provider meets your network, security and version needs |
| Scale of use | Several teams share the platform and need standards, access control and capacity planning | One team runs a small number of pipelines |
| Operations | Your team, or a contracted provider, will run and upgrade the scheduler, workers and metadata database | You want the smallest possible operational footprint |
Already using dbt? See how we orchestrate model runs on our dbt engineering page. Moving between self-managed Airflow and a managed service is itself an upgrade project: we never assume Dags will run unchanged.
ARCHITECTURE AND EXECUTORS
An Airflow 3 deployment has a scheduler, a Dag processor that parses pipeline code from a Dag bundle, an API server for the UI and REST API, and a metadata database. Workers and a triggerer are added as needed. Each executor and placement choice trades isolation, cost and operating effort differently. We choose them per workload, then prove recovery from a lost scheduler, worker or database in tested recovery exercises against objectives you approve.
| Decision | Typical approach | Trade-off we make explicit |
|---|---|---|
| Executor | LocalExecutor for small single-machine setups; CeleryExecutor or KubernetesExecutor for distributed work; several executors configured together where workloads differ | Simplicity and start-up speed vs. isolation and independent scaling |
| Task isolation | One pod per task with the KubernetesExecutor, or separate worker pools, for teams or code with different trust levels | Stronger isolation vs. more infrastructure to run |
| Dag delivery | Dag bundles from version control, with the Dag processor running as its own service | Controlled, versioned releases vs. a slower path for quick fixes |
| Metadata database | PostgreSQL or MySQL, sized, backed up and cleaned on a schedule | Database effort vs. scheduler health and upgrade time |
| Deferred work | A triggerer for deferrable operators that wait on external systems | One more service vs. worker slots freed while tasks wait |
OPERATIONS
Coding standards, import-time checks and automated tests run before Dags reach production, so one broken file does not stall the platform.
Feature and patch upgrades for core and providers, with upgrade checks, a database backup, staging rehearsal and a rollback decision point.
Pools, queues and executor settings tuned from measured task load, so critical pipelines get capacity when they need it.
Scheduler, worker and metadata database health with actionable alerts, plus database backups, cleanup and restore drills.
UPGRADE
Delivered through our migration and modernization practice, from Airflow 2 or from other schedulers.
Airflow 2 reached end of life on 22 April 2026. Start with an upgrade check on your own Dags.
USE CASES
Ingestion, transformation and table maintenance across warehouses and lakehouses, including scheduled dbt runs.
Feature preparation, training, evaluation and batch inference steps, with each run tracked and repeatable.
Pipelines that start when upstream data or an external event arrives, rather than only on a clock.
SUPPORT ROUTES
Apache Airflow is open source. Several companies also offer commercial routes: Astronomer offers commercial Airflow products, including its Astro platform; Amazon Web Services offers Amazon Managed Workflows for Apache Airflow (Amazon MWAA); and Google offers Cloud Composer, which it is renaming Managed Service for Apache Airflow. Bassirah is not a partner of Astronomer, Amazon Web Services or Google Cloud. We support self-managed Airflow independently and are clear about where our commitments end.
| Decision factor | Community only | Bassirah independent support | Managed service or vendor contract |
|---|---|---|---|
| Who operates Airflow | Your team | Your team with Bassirah engineering, or Bassirah under a managed scope | The provider runs the platform; your team owns the Dags, under the provider's terms |
| Help with incidents | Public community channels, no response commitment | Response targets per your agreement | Provider support under its own terms |
| Code defects | Reported by you to the project | Reproduced, worked around and reported upstream by Bassirah | Handled by the provider under its terms |
| Upstream fixes and releases | On the project's timeline | On the project's timeline; we track them, test them and apply them through change control | Versions and upgrade timing set by the provider |
| Commercial products | Not included | Not included: Bassirah does not resell Astronomer, Amazon MWAA or Cloud Composer | Contracted directly with the provider |
If you need a managed Airflow service or vendor support, that is contracted with the provider you choose. How cases, severities and response targets work for our independent support is set out in our open-source support model. For fully contracted operations, see Managed Services.
WORKS WITH
Airflow coordinates work that other platforms perform. We connect it to transformation, processing, metadata and storage where each fits, and test every integration first.
Transformation models and tests whose scheduled runs and dependencies Airflow can orchestrate.
Batch and large-scale processing jobs that Airflow submits, tracks and retries.
Metadata and lineage for the pipelines Airflow runs, where the integration is configured and tested.
S3-compatible object storage for the data, files and lakehouse tables that pipelines read and write.
Yes. Apache Airflow is a project of The Apache Software Foundation, released under the Apache License 2.0. Astronomer, Amazon MWAA and Google Cloud Composer are commercial routes from other companies. Bassirah supports self-managed Airflow and is not endorsed by The Apache Software Foundation or by those companies.
Not by the project. Airflow 2 entered limited maintenance on 22 October 2025 and reached end of life on 22 April 2026; its last release is 2.11.2. The project issues security advisories only for release lines that still receive releases, so Airflow 2 deployments should plan the move to Airflow 3.
Often not. Airflow 3 removes SubDAGs, SLAs, several context variables such as execution_date, and direct metadata database access from task code, and it changes some defaults. The project’s Ruff upgrade rules find many issues, and we test every Dag in staging before cutover.
We work on the Airflow 3 release line, with 3.3 as the default target, and we plan upgrades for deployments on earlier Airflow 3 releases and on Airflow 2. Pre-release versions, such as the 3.4 betas, are for testing only. The versions in scope are confirmed in each statement of work.
No. Bassirah is not a partner of Astronomer, Amazon Web Services or Google Cloud, and does not provide vendor support, vendor security fixes or vendor escalation. Our services are independent engineering and support, with scope and response targets defined in your agreement. Code fixes and releases follow the project’s timeline. See our support model.
A managed service suits teams that accept the provider’s versions, networking and upgrade timing. Self-managed Airflow suits teams that need control over executors, isolation and upgrade timing. We compare both against your requirements before you commit.
Tell us about your Dags, executors and upgrade plans. We will scope an assessment and a practical next step.
Operations & Reliability
Airflow 2 reached end of life in April 2026. What Airflow 3 changes, which version to target, and a planning sequence that keeps upgrade risk…
Apache Airflow
Read article : Airflow 2 End of Life: Planning the Upgrade to Apache Airflow 3→
