Analytics engineering lab on financial crime

Overview

This is a personal learning and demonstration project. It simulates, with synthetic data, the kind of analytical environment used to study customer risk and financial crime. It is not a deployment for a real client and contains no information from any company.

Problem and goal

I wanted a place to practise and show how a complete data project is structured: layered modelling, quality tests, repeatable transformations and orchestrated execution. I chose a domain I know from experience — customers, transactions and risk signals — and synthetic data so I can show it without compromising anyone’s data.

Technical architecture

The project follows a layered organisation:

• Base: sources loaded almost untouched, with consistent names and types.
• Core: business entities — customers and transactions — with well-defined keys and rules.
• Marts: tables prepared for specific analyses.

Models are written in SQL with dbt, which manages dependencies, tests and documentation. Execution is orchestrated with Airflow and automatic checks run with GitHub Actions.

I’m happy to walk through the implementation details (engines, models and specific parameters) in a conversation.

Technologies

• Python for synthetic data generation and utilities.
• SQL and dbt for transformations, tests and documentation.
• DuckDB and BigQuery as analytical engines.
• Airflow for orchestration.
• GitHub Actions for continuous integration.

Implementation

The dataset contains more than 250,000 synthetic transactions linked to customers. From them the layers above are built and quality tests run: uniqueness and not-null on keys, relationships between tables and simple business rules. The aim is a reproducible process: the same run always gives the same results. The code is private and I am happy to explain it in detail.

Challenges and decisions

• Synthetic rather than real data. It allows publishing the project but requires generating data with a believable structure. Analytical conclusions must not be extrapolated to real data.
• Explicit layers. More effort at first, but it separates source changes from business-rule changes and makes each part easier to test.
• Tests from the first model. Catching a problem in the base layer is much cheaper than discovering it in a final report.

Results and evaluation

Verifiable outcome: a reproducible environment with more than 250,000 synthetic transactions, a three-layer architecture, quality tests and orchestrated transformation flows. No business metrics are published because the data is synthetic.

What’s next

Document the model catalogue and add more test scenarios. If any of this is useful for your case, the data engineering service describes how I apply it.