Hi J***y,
Please find below a sanitized snippet of our Airflow “DAG factory” module and a sample feature‐prioritization scorecard. I hope these give you the visibility you need ahead of our Tuesday call.
1. DAG Factory Snippet (factory.py)
```python
from datetime import timedelta
from airflow import DAG
from airflow.operators.python_operator import PythonOperator
from airflow.utils.dates import days_ago
from hooks.alerts import failure_alert_hook
class DAGFactory:
# Shared default_args for all DAGs
default_args = {
'owner': 'data_team',
'depends_on_past': False,
'email_on_failure': True,
'email_on_retry': False,
'email': ['d***@***.com'],
'retries': 2,
'retry_delay': timedelta(minutes=10),
'on_failure_callback': failure_alert_hook
}
@staticmethod
def create_dag(dag_id, schedule_interval, start_date=days_ago(1), dag_kwargs=None):
args = DAGFactory.default_args.copy()
if dag_kwargs:
args.update(dag_kwargs)
dag = DAG(
dag_id=dag_id,
default_args=args,
description='Auto-generated DAG',
schedule_interval=schedule_interval,
catchup=False,
tags=['production']
)
# Example templated task
with dag:
run_etl = PythonOperator(
task_id='run_etl_job',
python_callable=lambda **ctx: print("Running ETL"),
retries=args['retries']
)
return dag
```
Key points:
- default_args defined in one place for owner, retries, retry_delay, alerts, etc.
- Each DAG calls `DAGFactory.create_dag(...)` and only specifies its own task graph.
- We use dev/prod separation with CI/CD deployments from GitLab.
2. Sample Quarterly Workshop Scorecard
| Feature | SHAP Importance | Historical Lift | Effort (0,5) | Weighted Score |
|----------------------------|-----------------|-----------------|--------------|-------------------------------|
| New Customer Recency Index | 0.42 | 1.7× | 2 | (0.42×0.4)+(1.7×0.4)+((5,2)/5×0.2) ≈ 0.65 |
| Promo Exposure Count | 0.25 | 1.3× | 1 | ≈ 0.58 |
| Location Footfall Estimate | 0.18 | 1.2× | 3 | ≈ 0.45 |
| Competitor Proximity Score | 0.15 | 1.1× | 2 | ≈ 0.40 |
Weights default to SHAP 40% / Lift 40% / (5,Effort) 20%, with ±10% flexibility if a feature is mission-critical or time-sensitive.
Let me know if you’d like any deeper context or tweaks before our Tuesday 10:00 am AEST call with S***n. Looking forward to diving in!
Best,
J***e
HR Recruiter, StrategyBrain