About
Hi, I'm Alfio.
I'm a data engineer based in the United Kingdom. I began my career in Business Intelligence and moved down the stack into data engineering, where I now work with Big Data, Apache Spark, Databricks, Azure Data Factory, Airflow, Machine Learning and MLOps.
Why "Composite Intelligence"?
I bought compositeintelligence.net because of a small equation I couldn't stop thinking about: AI + BI = CI. Artificial Intelligence plus Business Intelligence equals Composite Intelligence.
Having worked on both sides, I see BI and AI as two halves of the same job. BI gives organisations numbers they can trust. AI and machine learning give them foresight and the ability to act. On their own, each falls short: dashboards that never suggest a next step, or models built on numbers nobody agrees with.
Data engineering is the glue. Pipelines, Lakehouse platforms, orchestration and MLOps are what make it possible for a governed metric and a model's prediction to sit side by side and feed the same decision. That's the work I enjoy most, and what this site is about.
The path so far
- 01
Business Intelligence Developer
Where it started: data models, warehouses, reports and dashboards. I learned that a number is only useful if people trust it, and that trust comes from definitions, testing and lineage.
- 02
Data Engineer
Today: building the platforms underneath. Spark and Databricks for large-scale processing, Azure Data Factory and Airflow for orchestration, and ML and MLOps pipelines that take models from notebook to production.
- 03
Composite Intelligence
What I am working towards: systems where BI-grade trust and ML-driven foresight live on the same platform, with this site as the public lab notebook.
Toolbox
The tools and practices I use day to day, grouped roughly by layer.
Processing
- Apache Spark
- PySpark
- Spark SQL
- Databricks
- Delta Lake
Orchestration & integration
- Apache Airflow
- Azure Data Factory
- Databricks Workflows
ML & MLOps
- Machine Learning
- MLflow
- Model monitoring
- Feature pipelines
Foundations
- Python
- SQL
- Dimensional modelling
- Azure
- Git & CI/CD
Business Intelligence
- Data warehousing
- Semantic models
- KPI design
- Dashboards
What this site is for
- Writing: practical notes on Spark, Databricks, orchestration, Lakehouse design and MLOps, plus the occasional opinion piece.
- Projects: end-to-end builds with public code, so you can see how I approach real problems rather than take my word for it.
- Learning in public: writing things down is the best way I know to understand them properly.
Currently open to
- Data engineering roles: platform, pipelines, and ML infrastructure
- Conversations about Lakehouse, Spark, and MLOps architecture
- Collaborating on open source and community projects