Business Intelligence brings trust
Governed metrics, clean models, lineage and explainable numbers. It tells you what happened and why, and everyone can agree on the answer.
Composite Intelligence
AI + BI = CI
Artificial Intelligence plus Business Intelligence equals Composite Intelligence: data systems that are trusted like a BI report and as adaptable as a machine learning model.
I'm Alfio, a data engineer based in the United Kingdom. I started out in BI and now build data platforms and ML pipelines with Spark, Databricks and Airflow. This is where I write about that work and share projects you can run yourself.
The idea
Two disciplines, one system
BI and AI grew up as separate traditions. The interesting work happens where they overlap, and data engineering is what holds them together.
Artificial Intelligence brings foresight
Forecasts, anomaly detection, scoring and language models. It suggests what is likely to happen next and what could be done about it.
Composite Intelligence closes the loop
Reliable data platforms where trusted metrics feed models, models feed decisions, and outcomes flow back as data. That is the part I build.
Writing
Latest from the blog
Why Composite Intelligence: where BI ends and AI begins
AI + BI = CI. How the trust of Business Intelligence and the adaptability of Artificial Intelligence combine into one practical, closed loop.
- Composite Intelligence
- BI
- AI
- Machine Learning
Projects
What I'm building
End-to-end builds, each with public code and a write-up of the decisions behind it.
CI Loop: closed-loop analytics
The Composite Intelligence idea as a runnable demo: governed BI metrics, ML anomaly detection and forecasting, human-approved actions, and outcomes fed back as data.
- Python
- PySpark
- Delta Lake
- MLflow
- Airflow
- SQL
Medallion Lakehouse on Databricks
A reference implementation of an Azure Lakehouse: Data Factory ingestion, Auto Loader into Delta, bronze/silver/gold modelling, Unity Catalog governance and BI-ready gold tables.
- Databricks
- Delta Lake
- PySpark
- Azure Data Factory
- Unity Catalog
- Databricks Asset Bundles
Production ML pipeline with MLflow and Airflow
Taking a model from notebook to production: feature pipelines in Spark, experiment tracking and a model registry in MLflow, batch scoring orchestrated by Airflow, and drift monitoring.
- Apache Airflow
- MLflow
- PySpark
- scikit-learn
- Docker
- GitHub Actions
Toolbox
What I work with
- Apache Spark
- PySpark
- Spark SQL
- Databricks
- Delta Lake
- Apache Airflow
- Azure Data Factory
- Databricks Workflows
- Machine Learning
- MLflow
- Model monitoring
- Feature pipelines
- Python
- SQL
- Dimensional modelling
- Azure
- Git & CI/CD
- Data warehousing
- Semantic models
- KPI design
- Dashboards
Let's talk data
Hiring for a data engineering role, working on a Lakehouse or MLOps problem, or just want to compare notes? I'm always happy to talk.