Technical Reference Ebook ยท Confidential

Data Intelligence
Platform

Azure Databricks ยท Medallion Architecture ยท ML & Analytics

500KActive Customers
R2.3TAssets Under Mgmt
200Branch Network
12ML Models Live

Version 3.1  |  June 2025  |  Data Engineering Team  |  CONFIDENTIAL

Chapter 1

Executive Summary

The strategic vision, key outcomes, and platform metrics for Prime Capital Bank's unified data intelligence initiative.

Prime Capital Bank has embarked on a transformational journey to consolidate its fragmented data landscape into a single, governed, cloud-native data intelligence platform. Operating across 200 branches with 500,000 active retail, commercial, and corporate customers, and managing a portfolio exceeding R2.3 trillion in assets under management, the bank required a platform capable of meeting the dual demands of real-time operational intelligence and rigorous regulatory compliance.

The legacy environment comprised more than a dozen isolated system-of-record databases โ€” Temenos T24 core banking, Salesforce CRM, a proprietary card management system, Kondor+ treasury, Oracle Financials general ledger, and multiple third-party data feeds from Reuters, the South African Reserve Bank (SARB), and TransUnion credit bureau. Data resided in siloed on-premises servers, batch-transferred nightly through brittle ETL pipelines, with analytical workloads competing for OLTP resources and risk models running days behind the market.

Platform Vision

To build a single source of analytical truth for Prime Capital Bank โ€” a cloud-native, governed, ML-enabled data platform that eliminates data silos, accelerates regulatory reporting, and delivers actionable intelligence to every business unit within the SLA window.

Key Business Outcomes

The Data Intelligence Platform has delivered measurable outcomes across four strategic dimensions: operational efficiency, risk management, regulatory compliance, and customer intelligence.

99.8%Pipeline SLA
<2sAvg Query Time
40%Less Manual Reporting
R45MFraud Savings / Year
12Live ML Models
247Tables Catalogued
78TBData Under Mgmt
500Power BI Users

The two charts below break down the platform’s most critical headline numbers: how the bank’s four primary loan product categories (representing R312.6 billion of the R380 billion total loan book) are distributed, and how the bank’s capital ratios compare to SARB minimum requirements.

Figure 1.1 โ€” Loan Portfolio Composition & Capital Adequacy

The four primary loan categories total R312.6 billion โ€” 82% of the R380 billion gross loan book (the remaining 18% comprises credit cards, overdrafts, and revolving facilities). Home loans dominate at 40%, reflecting the bank's strategic focus on secured retail lending. Capital adequacy ratios comfortably exceed SARB minimum requirements, providing a substantial buffer for stressed scenarios and balance sheet growth.

Left: Loan book distribution by product (R billions). Right: Capital ratios vs SARB minimums (%). Source: Gold layer โ€” fact_loan_portfolio, regulatory_reporting.

Platform Components

The platform is built on Azure Databricks with a Medallion (Bronze โ†’ Silver โ†’ Gold) architecture hosted on Azure Data Lake Storage Gen2 (ADLS Gen2). Data transformation is managed through dbt Core running inside Databricks notebooks, with data quality enforced by Great Expectations. The governance layer is powered by Unity Catalog, providing column-level security, data lineage, and PII masking at the platform level. Machine learning workloads leverage MLflow and the Databricks Feature Store, with 12 production models serving real-time predictions via Azure API Management.

Business intelligence is delivered through Power BI Premium with 8 purpose-built dashboards serving executive leadership, risk management, compliance, branch operations, and treasury. Regulatory reporting โ€” including SARB BA700, IFRS 9 ECL provisioning, Basel III capital adequacy, and FICA/AML returns โ€” is fully automated within the platform, reducing the monthly close cycle from 5 business days to less than 6 hours.

Regulatory Coverage

The platform automates compliance reporting for SARB BA700, IFRS 9 Stage Classification and ECL provisioning, Basel III LCR/NSFR calculations, POPIA data subject rights management, and FICA/AML/CFT suspicious transaction reporting โ€” all within a single governed data lineage.

Strategic Importance

In an increasingly competitive South African banking landscape โ€” with challenger banks, digital wallets, and open banking pressures โ€” Prime Capital Bank's data platform provides a durable competitive advantage. The ability to compute real-time credit scores at point-of-application, detect fraudulent card transactions within 50 milliseconds, and generate SARB-ready regulatory returns at the push of a button positions the bank at the forefront of data-driven banking in sub-Saharan Africa.

The platform is designed for scale: the Databricks cluster configuration supports linear horizontal scaling to accommodate projected 5-year data growth of 300%, while the Unity Catalog governance model ensures that every new data asset inherits the same security, lineage, and quality standards established at platform launch.

Chapter 2

Architecture Overview

The Azure cloud-native Medallion architecture underpinning the platform.

The Prime Capital Bank Data Intelligence Platform is designed around the Medallion Architecture pattern โ€” a multi-layered data organisation framework that progressively refines raw source data through Bronze, Silver, and Gold quality tiers. This architecture, hosted entirely on Microsoft Azure, provides a clear separation of concerns between ingestion, transformation, and consumption, enabling independent scaling and governance at each layer.

Azure Architecture Diagram
Figure 2.1 โ€” Prime Capital Bank Azure Data Platform Architecture. Source systems on the left feed ADLS Gen2 storage zones (centre-left), processed by Databricks notebooks (centre), governed by Unity Catalog, and served to consumption layers (right).

Azure Services Deployed

Azure ServiceRoleSKU / TierJustification
ADLS Gen2Primary data lake storageLRS ยท Hot + Cool tiersHierarchical namespace, ACL-level security, Delta Lake compatible
Azure DatabricksCompute & transformationPremium ยท E8ds_v4 clustersPhoton engine, MLflow, Delta Lake native, Unity Catalog integration
Unity CatalogData governancePlatform-levelCentralised metadata, column masking, data lineage, RBAC
Azure Key VaultSecrets managementStandardConnection strings, API keys, certificates โ€” no plaintext secrets in code
Azure Event HubReal-time streamingStandard ยท 20 TUsKafka-compatible API for fraud detection streaming ingestion
Azure MonitorObservability & alertingLog Analytics workspacePipeline health, cluster metrics, SLA alerting
Power BI PremiumBI & dashboardsP2 capacityDirectQuery to Databricks, paginated reports for regulatory output
Azure SQLOperational reporting storeGeneral Purpose ยท 8 vCoresSub-second branch and customer lookup queries for ops teams
API ManagementML model servingStandard v2REST endpoints for credit scoring and fraud models, rate limiting
Azure Active DirectoryIdentity & access managementP2SSO, MFA, Conditional Access for all platform users

Data Flow Architecture

Source data enters the platform through two primary ingestion mechanisms. Batch ingestion uses Azure Data Factory (ADF) pipelines and Databricks Auto Loader to pull structured data from source systems on defined schedules โ€” typically hourly for core banking and card transactions, daily for HR and GL, and monthly for external credit bureau feeds. Streaming ingestion routes card transaction events and ATM activity through Azure Event Hub, processed by Databricks Structured Streaming notebooks into the Bronze layer within seconds.

RAW~50TB ยท As-is
BRONZE~38TB ยท Delta
SILVER~22TB ยท Clean
GOLD~8TB ยท Analytics

All data at rest is encrypted using AES-256 via Azure Storage Service Encryption. All data in transit is protected by TLS 1.3. Azure Private Endpoints ensure that ADLS Gen2 and Databricks communicate exclusively over the bank's private VNET, with no public internet exposure. Network Security Groups (NSGs) enforce allow-list-only ingress and egress rules for each subnet.

Security Posture

The platform achieved an ISO 27001-aligned security design with zero public endpoints. All compute clusters use customer-managed keys (CMK) via Key Vault, Managed Identity eliminates credential proliferation, and Unity Catalog enforces row-level and column-level security for every data access regardless of tool used.

Network & Identity Design

The Databricks workspace is deployed in a customer-managed VNET (VNet injection) with separate subnets for the driver and worker node pools. Azure Private Link connects Databricks to ADLS Gen2, Azure SQL, Key Vault, and Event Hub โ€” ensuring all data movement stays within the Azure backbone. All user access is mediated through Azure Active Directory with Privileged Identity Management (PIM) for Just-In-Time elevation of data engineer and DBA roles.

Chapter 3

Data Model โ€” Bronze Layer

Raw ingestion landing zone: schema preservation, incremental load, and audit trail.

The Bronze layer is the first tier of refinement in the Medallion architecture. Its primary purpose is to faithfully capture and persist data from source systems with the minimum possible transformation โ€” preserving the original schema, data types, and even known data quality issues for full traceability. Bronze is the system of record within the data lake; if a discrepancy arises between the platform and a source system, Bronze is the forensic reference.

Bronze Design Principle

Every record ingested into Bronze is immutable and auditable. Records are never deleted or overwritten โ€” only appended. Each record carries ingestion metadata: _ingestion_timestamp, _source_system, _source_file, _batch_id, and _record_hash for deduplication downstream.

Ingestion Mechanisms

Databricks Auto Loader (cloudFiles format) handles the majority of file-based ingestion from the Raw zone of ADLS Gen2. Auto Loader provides incremental processing with exactly-once semantics via a checkpoint mechanism stored in ADLS, making it resilient to cluster failures. Schema inference and evolution are managed automatically โ€” when a source system adds a new column, Auto Loader detects the schema change and uses column merging to extend the Bronze Delta table without data loss or pipeline failure.

For core banking and card system feeds arriving as structured database exports, Azure Data Factory orchestrates the movement from on-premises SQL Server instances to the Raw zone via a Self-Hosted Integration Runtime (SHIR) installed in the bank's data centre. For real-time card transactions, Databricks Structured Streaming consumes from Azure Event Hub using the Kafka consumer API.

Bronze Table Inventory

Table NameSource SystemRefreshEst. RowsFormat
bronze.core_banking.accountsTemenos T24Hourly1.8MDelta
bronze.core_banking.transactionsTemenos T24Streaming2.1BDelta
bronze.core_banking.loansTemenos T24Hourly380KDelta
bronze.core_banking.customersTemenos T24Daily500KDelta
bronze.crm.contactsSalesforceDaily620KDelta
bronze.crm.opportunitiesSalesforceDaily1.4MDelta
bronze.cards.card_transactionsCard ManagementStreaming950MDelta
bronze.cards.card_accountsCard ManagementHourly280KDelta
bronze.payments.eft_transactionsPayment GatewayNear-real-time420MDelta
bronze.payments.rtc_paymentsPayment GatewayStreaming85MDelta
bronze.treasury.fx_ratesKondor+Hourly4.2MDelta
bronze.gl.journal_entriesOracle FinancialsDaily220MDelta
bronze.hr.employeesSAP HCMDaily12KDelta
bronze.external.credit_bureauTransUnionMonthly500KDelta

The chart below visualises the relative scale of each Bronze source table, making the volume disparity immediately clear and highlighting which feeds drive the platform’s throughput requirements.

Figure 3.1 โ€” Bronze Layer โ€” Data Volumes by Source Table

The Bronze layer ingests data from 14 source tables spanning 7 source systems. Transaction and payments tables dominate by volume, with card transactions (950M rows) and EFT payments (420M rows) representing the bulk of streaming ingestion throughput. These high-volume streaming tables drive the platform's near-real-time fraud detection and liquidity monitoring capabilities.

Row counts in millions. Log scale applied due to 4-order-of-magnitude range between smallest (employees: 12K) and largest (transactions: 2.1B) tables. Source: Bronze layer metadata, June 2025.

Schema & Partitioning Strategy

Bronze tables are partitioned by _ingestion_date (date of load) rather than source business date, ensuring partition pruning works effectively even when source systems deliver late-arriving data. High-volume streaming tables (card transactions, EFT) are additionally partitioned by hour to enable efficient time-range queries during fraud investigation workflows.

Delta Lake's OPTIMIZE and ZORDER commands are scheduled nightly on high-query Bronze tables to compact small files produced by streaming micro-batch writes. The VACUUM command retains 30 days of Delta history, supporting point-in-time recovery and audit queries within the regulatory retention window.

Auto Loader Configuration (Example)

Bronze ingestion notebooks use spark.readStream.format("cloudFiles").option("cloudFiles.format", "parquet").option("cloudFiles.schemaLocation", schema_path).load(raw_path) with .writeStream.format("delta").option("checkpointLocation", checkpoint_path).outputMode("append").trigger(availableNow=True) for triggered incremental runs โ€” combining the efficiency of batch processing with the correctness guarantees of streaming checkpoints.

Chapter 4

Data Model โ€” Silver Layer

Cleansed, standardised, and enriched data ready for domain modelling.

The Silver layer transforms raw Bronze data into a clean, conformed, business-meaningful dataset. Silver applies the bank's master data management (MDM) rules, data quality frameworks, and business logic โ€” producing a set of tables that are source-agnostic and conforming to the bank's canonical data model. Silver is the primary input for the Gold analytics layer and for ad-hoc data science exploration.

Transformation Categories

Silver transformations are organised into four categories, each implemented as a discrete set of dbt models executed in dependency order by Databricks Workflows:

Data Quality Framework

Silver quality gates are enforced using Great Expectations suites embedded in each transformation notebook. Quality checks include:

Check TypeExampleAction on Failure
Not Nullcustomer_id, transaction_dateQuarantine to silver.quarantine table
Referential Integrityaccount_id exists in dim_accountAlert + quarantine, pipeline continues
Range Checktransaction_amount between -R10M and R10MFlag for manual review
UniquenessOne record per (account_id, transaction_date, seq_no)Dedup and alert
Format ValidationSA ID number passes Luhn-mod-10 checkMask and quarantine
FreshnessMax ingestion_timestamp within last 2 hoursPagerDuty alert to on-call engineer
Completeness>99% of expected daily transaction volume receivedEscalation to source system owner
Quality Metrics โ€” Monthly Average

99.94% of records pass all Silver quality gates without quarantine. The quarantine rate of 0.06% is dominated by late-arriving GL corrections and test transactions from the card system's UAT environment, both of which are filtered deterministically by source system flag.

Figure 4.1 โ€” Silver Data Quality Gate Results

Great Expectations quality suites enforce 7 categories of validation before any record is promoted from Bronze to Silver. The 99.94% pass rate reflects the high data quality of the bank's core systems post-cleansing. The 0.06% quarantine rate breaks down into five failure types โ€” late GL corrections (38%) and card UAT test records (31%) dominate, followed by missing reference data (14%), format errors (11%), and duplicate keys (6%). All categories are filtered deterministically; no records are permanently deleted โ€” quarantined rows are available for manual review and retrospective promotion.

Left: Overall pass/quarantine rate (monthly average). Right: Quarantine breakdown by failure type. Source: Great Expectations validation results, silver.quarantine table.

SCD Type 2 Implementation

Customer, account, and employee dimensions are implemented as Slowly Changing Dimension Type 2 (SCD2) entities at the Silver layer, using the MERGE INTO command on Delta Lake tables. When a source record changes (e.g., a customer changes their residential address or credit segment), a new version row is inserted with an updated scd2_effective_date, the previous row's scd2_expiry_date is set to the change date minus one day, and the is_current_flag is updated accordingly. This approach preserves full change history for regulatory audit and enables point-in-time portfolio analysis โ€” critical for IFRS 9 stage migration reporting.

Silver Table Summary

The Silver layer comprises 42 tables across 6 domain schemas: silver.customers, silver.accounts, silver.transactions, silver.loans, silver.treasury, and silver.reference. Total compressed storage is approximately 22TB across all partitions, with a 60-day active query window supported by Delta Lake's time-travel capability and ADLS Cool tier lifecycle management for data older than 90 days.

Chapter 5

Data Model โ€” Gold Star Schema

Analytics-ready star schema: dimension tables, fact tables, and query optimisation strategy.

The Gold layer is the analytics-serving tier of the platform โ€” purpose-built for high-performance querying by Power BI, Azure SQL, data scientists, and regulatory reporting engines. Gold implements a classic Kimball star schema with denormalised dimension tables and centralised fact tables, partitioned and Z-ordered for sub-second query performance at petabyte scale.

Gold Star Schema ERD
Figure 5.1 โ€” Gold Layer Star Schema ERD. Three central fact tables surrounded by eleven dimension tables. All foreign key relationships shown. SCD Type 2 dimensions maintain full history.

Fact Tables

fact_transaction

The highest-grain fact table in the schema, storing 2.1 billion individual financial transactions spanning all channels: teller, ATM, online banking, mobile, POS, and EFT. Each row represents one debit or credit event against an account. Key measures include transaction_amount_zar, running_balance_zar, fraud_score, and aml_alert_flag. Partitioned by transaction_date and Z-ordered by customer_sk, date_sk to optimise customer-level time-series queries used in the Customer 360 dashboard.

fact_loan_portfolio

Monthly snapshot of the bank's R380 billion loan book โ€” retail mortgages, vehicle finance, personal loans, credit cards, and commercial credit facilities. Each row represents one loan account at month-end, with IFRS 9 Stage classification (1/2/3), PD score, LGD estimate, and ECL provision in ZAR. Partitioned by snapshot_month for regulatory reporting. Total row count: approximately 85 million rows spanning 36 months of history.

fact_account_balance

Daily snapshot of every active account's opening balance, closing balance, interest accrued, and days-past-due status โ€” 730 million rows covering 36 months of daily snapshots across 1.8 million accounts. Used primarily by the Treasury dashboard for liquidity analysis and the Branch Performance dashboard for deposit book tracking. Partitioned by balance_date and Z-ordered by branch_sk, product_sk.

Dimension Tables

DimensionSCD TypeRow CountKey Attributes
dim_customerSCD2~500K currentSegment, credit rating, FICA status, risk band
dim_accountSCD2~1.8MAccount type, product, status, credit limit
dim_productSCD1~850Product family, regulatory class, risk weight
dim_dateStatic10,957Calendar, fiscal calendar, SA public holidays
dim_branchSCD2200Region, province, type, manager, coordinates
dim_employeeSCD2~12KJob title, department, branch, employment status
dim_channelStatic18Channel type, is_digital, platform
dim_currencyDaily refresh~45ISO code, exchange rate to ZAR, SARB source
dim_geographyStatic~4,200Postal code, suburb, province, income decile
dim_risk_categorySCD1~120Basel class, IFRS9 stage, PD/LGD band
dim_merchantMonthly~250KMCC code, risk level, AML classification

Partitioning & Performance

All Gold tables use liquid clustering (Databricks Runtime 13+) on the highest-cardinality filter columns, replacing static partitioning for dimension tables. Fact tables retain date-based partitioning for partition pruning in regulatory reporting queries, supplemented by Z-order on the most common join keys. The Photon vectorised query engine typically delivers 3โ€“8x query speedup over standard Spark SQL for analytical aggregations on Gold tables.

Performance Benchmark

Power BI DirectQuery against fact_transaction with a 3-month filter window and branch grouping returns in 1.4 seconds average on a Databricks SQL Serverless warehouse (2X-Large). Full 36-month regulatory queries complete in under 45 seconds on a dedicated job cluster.

Chapter 6

dbt Transformation Layer

dbt Core project structure, testing strategy, and lineage management within Databricks.

dbt Core (data build tool) is the transformation engine for the Silver and Gold layers. Running natively inside Databricks notebooks via the dbt-databricks adapter, dbt provides SQL-first transformations with built-in dependency management, automated documentation, and a rich testing framework โ€” replacing hundreds of ad-hoc Spark notebooks with a structured, version-controlled, testable codebase.

Project Structure

-- dbt project layout (prime_capital_bank) models/ staging/ -- 1:1 mapping from Bronze, minimal transforms stg_core_banking__accounts.sql stg_core_banking__transactions.sql stg_crm__customers.sql stg_cards__transactions.sql stg_payments__eft.sql stg_gl__journal_entries.sql -- 14 staging models total intermediate/ -- Business logic, joins, SCD2 logic int_customer_master.sql int_account_enriched.sql int_loan_portfolio_classified.sql int_transaction_fraud_scored.sql int_ecl_provision_calculated.sql -- 18 intermediate models total marts/ -- Gold star schema output dimensions/ dim_customer.sql -- SCD2 merge logic dim_account.sql dim_product.sql dim_date.sql dim_branch.sql facts/ fact_transaction.sql fact_loan_portfolio.sql fact_account_balance.sql regulatory/ rpt_ifrs9_ecl.sql rpt_sarb_ba700.sql rpt_basel_lcr.sql -- 22 mart models total tests/ -- Schema + custom data tests macros/ -- Reusable Jinja macros snapshots/ -- SCD2 snapshot definitions seeds/ -- Reference data CSVs (risk codes, currencies)

Testing Strategy

dbt tests are categorised into four types, with 1,847 total tests across the project:

Lineage & Documentation

dbt generates a fully interactive lineage DAG documenting every model, its upstream sources, downstream consumers, column descriptions, and test results. This lineage is surfaced in Unity Catalog, allowing data consumers to trace any Gold metric back to its Bronze source record โ€” a requirement for IFRS 9 audit trails and SARB regulatory inspections. All 54 models and 847 columns are documented with business-friendly descriptions maintained in schema.yml files, auto-published to the dbt documentation site hosted on Azure Static Web Apps.

dbt Run Statistics (Daily)

Full dbt build: 54 models ยท 1,847 tests ยท average runtime 22 minutes on a 4-node E8ds_v4 cluster ยท test pass rate 100% required before Gold tables are promoted to production ยท failures trigger automatic Slack notification and pipeline hold.

Chapter 7

ML Models

Twelve production machine learning models covering credit risk, fraud detection, AML, and customer intelligence.

The ML capability of the Data Intelligence Platform represents the bank's most significant analytical investment. Twelve models are deployed to production via MLflow Model Registry on Databricks, served through Azure API Management as REST endpoints for real-time scoring and as batch jobs for portfolio-level calculations. All models are governed by the bank's Model Risk Management (MRM) Policy, requiring independent validation, champion-challenger testing, and quarterly performance reviews.

PD Model (Basel IRB)

0.94
Gini Coefficient

XGBoost classifier predicting 12-month probability of default. 180 features from bureau, transaction, and behavioural data. Monthly batch scoring across full loan book.

LGD Model (IFRS 9)

0.89
Rยฒ Score

Gradient Boosting Regressor estimating loss given default. Collateral valuation, recovery history, and product type features. IFRS 9 ECL calculation dependency.

Fraud Detection

0.97
AUC-ROC

Real-time GBM model scoring card transactions within 50ms. Trained on 3 years of confirmed fraud labels. Saves approximately R45M/year in prevented fraud losses.

AML Scoring

0.91
AUC-ROC

Unsupervised + supervised hybrid model for suspicious transaction detection. Graph-based network analysis for mule account identification. FICA STR automation.

Figure 7.1 โ€” ML Model Performance Metrics

All four production models exceed the bank's minimum AUC-ROC threshold of 0.80 ("Good" tier), with the Fraud Ensemble achieving near-best-in-class performance at 0.97. The AUC-ROC metric captures the model's ability to distinguish between positive and negative cases across all decision thresholds โ€” a score of 1.0 is perfect, 0.5 is random. Precision and Recall are balanced via the F1 score; the fraud model prioritises Recall (catching actual fraud) while accepting a slightly higher false positive rate to minimise financial losses.

Left: AUC-ROC (PD, Fraud, AML) / Rยฒ (LGD) โ€” PD=0.97 (Gini 0.94), LGD Rยฒ=0.89, Fraud=0.97, AML=0.91. Gold dashed line = 0.90 preferred tier (absolute floor 0.80 โ€” all models exceed both thresholds). Right: Precision, Recall, F1 โ€” all models 0.876โ€“0.963. FY 2025 data, June 2025.

Figure 7.2 โ€” PD Model ROC Curve (AUC = 0.97)

What: The Receiver Operating Characteristic (ROC) curve plots the model's True Positive Rate (TPR โ€” correctly identified defaults) against the False Positive Rate (FPR โ€” non-defaults wrongly flagged) across every possible decision threshold from 0 to 1. The area under the curve (AUC = 0.97) summarises the model's overall discrimination ability in a single number; the shaded region is the discrimination gain over a random classifier (the diagonal dashed line).
Why: The SARB Prudential Authority and Basel Committee require documented discrimination testing for IRB-approved PD models. AUC โ‰ฅ 0.75 is the minimum for IRB approval; 0.97 places Prime Capital Bank comfortably in the "Excellent" tier. The ROC curve also shows that at a 5% FPR operating point the model already detects 91% of future defaulters โ€” critical for proactive provisioning under IFRS 9.
Where: Computed in 04_ml_credit_scoring.py using 5-fold stratified cross-validation. Logged as an MLflow artifact (PNG + CSV) under pd_model_validation/roc_curve and reproduced monthly in the Credit Risk Power BI dashboard.
How: XGBoost classifier, SMOTE oversampling for class imbalance (3.2% default rate). sklearn.metrics.roc_curve applied to out-of-fold probability scores; AUC via trapezoidal integration.

ROC curve โ€” PD XGBoost model, AUC = 0.97. Shaded teal = discrimination gain above random. Operating point: 5% FPR โ†’ 91% TPR. Dashed diagonal = random classifier. Source: 04_ml_credit_scoring.py, MLflow run, June 2025.

Figure 7.3 โ€” PD Model Feature Importance: Top 10 SHAP Values

What: SHAP (SHapley Additive exPlanations) values measure each feature's average absolute contribution to the model's prediction across the full 500,000-customer loan book. A higher bar means that feature moves the model's output more โ€” on average โ€” whether it increases or decreases the predicted probability of default. The chart shows the top 10 most influential features out of 180 total.
Why: POPIA's fairness principle and the bank's Model Risk Management Policy require that credit decisions be explainable to regulators, auditors, and affected customers. SHAP values provide a legally defensible, mathematically rigorous explanation for why a specific customer received a high or low PD score โ€” enabling the bank to contest spurious explanations and detect potential proxy discrimination (e.g., if a geographic feature inadvertently proxies for race).
Where: SHAP values are computed in 04_ml_credit_scoring.py and stored as a MLflow artifact. The top-10 summary chart is published to the Credit Risk Power BI dashboard and included in the quarterly model performance review pack submitted to the Model Risk Committee.
How: TreeSHAP algorithm (exact, O(TLD) complexity) applied to the XGBoost model against a 10,000-record background dataset. Mean absolute SHAP value computed per feature and ranked descending.

Horizontal bars โ€” mean |SHAP| value per feature (higher = more impactful). Payment recency and payment ratio dominate; bureau score ranks 3rd. Geography (branch region) has lowest individual impact of the top 10. Source: MLflow SHAP artifact, June 2025.

Figure 7.4 โ€” PD Model Calibration: Predicted vs Actual Default Rate by Score Decile

What: The loan book is ranked by predicted PD score and split into 10 equal-sized deciles (D1 = lowest-risk 10%, D10 = highest-risk 10%). The bars show the model's mean predicted default probability per decile; the gold line shows the observed (actual) default rate for loans in that decile over the subsequent 12-month outcome window. A well-calibrated model has bars and line closely tracking throughout all deciles.
Why: Calibration is the cornerstone of the Basel IRB use test. The bank must demonstrate to SARB that the model's scores translate accurately to real-world default frequencies โ€” otherwise the ECL provisions computed from those scores will either over- or under-state the bank's true credit losses, distorting reported profitability and capital adequacy. Calibration RMSE > 0.5 percentage points triggers an automatic model redevelopment review.
Where: Monthly backtesting in 04_ml_credit_scoring.py comparing June 2024 PD scores against June 2025 realised defaults from the loan servicing system. Results logged to MLflow and reviewed by the Model Risk team before each SARB BA210 submission.
How: Loans ranked by PD score โ†’ split into 10 equal-count deciles โ†’ actual defaults counted per decile โ†’ observed rate vs mean predicted PD plotted. Calibration RMSE = 0.18 pp for this run (well within the 0.5 pp tolerance).

Bars = predicted PD (%). Gold line = actual observed default rate (%) โ€” 12-month outcome window. Tight tracking across all 10 deciles confirms calibration quality. Calibration RMSE = 0.18 pp. Source: 04_ml_credit_scoring.py backtest, June 2025.

IFRS 9 ECL Model

The Expected Credit Loss (ECL) model is the regulatory centrepiece of the ML suite. It combines the PD, LGD, and Exposure at Default (EAD) models to compute the three-stage ECL provision required under IFRS 9 โ€” Financial Instruments. Stage 1 accounts (performing) receive a 12-month ECL; Stage 2 accounts (significant increase in credit risk) receive a lifetime ECL; Stage 3 accounts (credit-impaired) receive a lifetime ECL with individually assessed components for material exposures exceeding R5 million.

The ECL pipeline runs monthly โ€” triggered 2 business days before month-end close โ€” producing provisioning entries for the Oracle Financials GL, regulatory disclosures for the SARB BA210 return, and IFRS 9 note disclosures for the annual financial statements. Total ECL provision as at the most recent calculation: R4.8 billion against a gross loan exposure of R380 billion (1.26% provision coverage ratio).

Model Governance Requirements

All production models require: independent validation by the Model Risk team before deployment; Databricks MLflow experiment tracking with full parameter and metric logging; champion-challenger live comparison for 90 days post-deployment; quarterly backtesting against realised outcomes; and annual full model redevelopment review. Model performance degradation below defined AUC thresholds triggers automatic fallback to the champion model.

Fraud Detection โ€” Real-Time Architecture

The fraud model operates in a sub-50ms latency inference pipeline. Card transaction events arrive at Azure Event Hub, are consumed by a Databricks Structured Streaming job, enriched with the customer's Feature Store features (30-day spend velocity, merchant category history, device fingerprint), and scored by the MLflow-served GBM model. Scores above 0.85 trigger an immediate card hold and SWIFT message to the acquiring bank. Scores between 0.65 and 0.85 are flagged for the Fraud Operations team in their real-time dashboard. In 2024, the model prevented R45.2 million in fraudulent transactions with a false positive rate of 0.8% โ€” below the industry benchmark of 1.5%.

Feature Store

The Databricks Feature Store contains 340 pre-computed features organised into 12 feature tables, serving all 12 production models. Features are computed on scheduled Databricks jobs (5-minute cadence for fraud features, hourly for credit features, daily for AML behavioural features) and versioned with point-in-time correctness โ€” preventing training-serving skew by ensuring models trained on historical features see identical feature values during batch scoring of historical periods.

Chapter 8

Regulatory Compliance

Automated compliance for SARB, IFRS 9, Basel III, POPIA, FICA, and AML/CFT frameworks.

Prime Capital Bank operates under one of the most demanding regulatory environments in Africa. As a Systemically Important Financial Institution (SIFI) designated by the South African Reserve Bank (SARB), the bank is subject to the Prudential Authority's full suite of regulatory requirements. The Data Intelligence Platform automates the generation, validation, and submission of all prudential regulatory returns, reducing the compliance team's data preparation burden by 78% and eliminating the manual errors that previously triggered SARB Directive Letters.

BA700Automated Monthly
IFRS 9ECL Automated
Basel IIILCR/NSFR Daily
POPIAPII Governance
FICAAML/STR Auto
6hrsMonthly Close
Figure 8.1 โ€” Basel III Capital & Liquidity Ratios vs SARB Minimums

Prime Capital Bank maintains capital and liquidity ratios significantly above SARB prudential minima. The CET1 ratio of 14.2% provides a 7.2 percentage-point buffer above the 7.0% minimum requirement, giving the bank substantial capacity to absorb stress losses and support balance sheet growth without breaching regulatory floors. The LCR of 128.4% means the bank holds 28.4 percentage points of excess high-quality liquid assets above the 100% Basel III minimum โ€” critical for maintaining depositor confidence during market stress.

Grouped bar chart: actual ratios (teal) vs SARB minimum requirements (gold). All ratios comply. Source: Databricks regulatory_reporting notebook, June 2025.

SARB BA700 โ€” Risk Data Aggregation

The BA700 return is the SARB's primary prudential data collection, covering credit risk exposures, capital adequacy, liquidity, and large exposures. The platform generates the BA700 XML submission from a dedicated dbt mart (gold.regulatory.rpt_sarb_ba700), populated from the credit risk Gold tables. The submission is validated against the SARB's published XML schema definition (XSD) before upload to the SARB's BI reporting portal. The platform achieves T+2 submission (two business days after month-end) against the regulatory deadline of T+10.

Basel III Capital Adequacy

The platform computes Liquidity Coverage Ratio (LCR) and Net Stable Funding Ratio (NSFR) on a daily basis using the Gold layer's account balance and treasury position data. Risk-weighted asset (RWA) calculations for credit risk use the Standardised Approach weights stored in dim_risk_category.regulatory_weight, with Internal Ratings-Based (IRB) approach RWA calculated from the PD model outputs for the qualifying retail and corporate portfolios. The bank maintains a Common Equity Tier 1 (CET1) ratio of 14.2% against the SARB's effective minimum of 7.0% (4.5% CET1 floor plus the 2.5% capital conservation buffer), providing a 7.2 percentage-point headroom โ€” consistent with Figure 8.1 below.

POPIA โ€” Data Privacy

The Protection of Personal Information Act (POPIA) compliance is implemented at the platform layer through Unity Catalog tags and column masking policies. All columns containing personal information (SA ID numbers, full names, contact details, biometric data) are tagged with the pii_classification Unity Catalog tag at three levels: PII_HIGH (ID numbers, biometrics), PII_MEDIUM (names, contact details), and PII_LOW (demographic aggregates). Column masking functions applied via Unity Catalog policies ensure that users without the pii_reader privilege see tokenised or masked values in query results โ€” enforced at the Delta Lake level, not application level.

FICA & AML/CFT

The Financial Intelligence Centre Act (FICA) compliance layer includes automated Know Your Customer (KYC) completeness checks, enhanced due diligence (EDD) flagging for high-risk customers, and automated Suspicious Transaction Report (STR) generation. The AML scoring model identifies suspicious transaction patterns โ€” structuring, layering, round-trip transactions, and dormant account activation โ€” and generates STR drafts pre-populated with the FIC's required fields, routed to the Compliance team for review and submission to the Financial Intelligence Centre within the 15-day statutory window.

Compliance Automation Impact

Monthly close cycle reduced from 5 business days to 6 hours. SARB return preparation reduced from 14 person-days to 0.5 person-days (validation and sign-off only). STR preparation time reduced from 4 hours per report to 12 minutes. Zero SARB Directive Letters received since platform go-live (vs. 3 in the prior 24-month period).

Chapter 9

Pipeline Orchestration

Databricks Workflows, scheduling, SLA management, monitoring, and disaster recovery.

All data pipeline execution is orchestrated by Databricks Workflows โ€” the native job orchestration engine integrated with the Databricks workspace. The platform runs 47 production pipelines across four scheduling tiers: streaming (continuous), near-real-time (5-minute), hourly, and daily. Databricks Workflows provides dependency-aware DAG scheduling, retry logic, cluster auto-provisioning, and native integration with Azure Monitor for alerting.

Pipeline Hierarchy

PipelineScheduleSLACluster TypeDependencies
Fraud StreamingContinuous<50ms latencyStreaming (4 workers)Event Hub
Bronze โ€” CardsEvery 5 minT+10minJob (3 workers)Event Hub checkpoint
Bronze โ€” Core BankingHourlyT+45minJob (4 workers)ADF copy job
Silver โ€” All domainsHourlyT+90minJob (6 workers)Bronze pipelines
Gold โ€” DimensionsDaily 02:0005:00Job (8 workers)Silver complete
Gold โ€” FactsDaily 05:0008:00Job (12 workers)Gold dims complete
ML โ€” Credit ScoringMonthly T-218hrsGPU (4ร— A100)Gold loan data
Regulatory ReportsMonthly T+1T+3 businessJob (8 workers)Gold + ML complete
Figure 9.1 โ€” Daily Pipeline Execution Gantt โ€” 9-Task DAG

The production daily pipeline executes as a 9-task Directed Acyclic Graph (DAG) orchestrated by Databricks Workflows. The pipeline begins at 00:30 with cluster setup and Bronze ingestion, progresses through Silver cleansing, then fans out into three parallel ML scoring tasks (Credit, Fraud, AML) running concurrently with Gold star schema build (25 min) and three parallel ML scoring tasks. AML Risk Scoring is the longest individual stage at 30 minutes. Regulatory reporting and the master orchestrator complete the pipeline by approximately 02:15, comfortably within the 05:00 Gold SLA.

Gantt chart โ€” X axis: minutes elapsed from pipeline trigger (00:30 SAST). Parallel tasks shown at same Y position. Source: Databricks Workflow run history, June 2025.

SLA Monitoring & Alerting

Pipeline SLA breaches trigger a three-tier alerting cascade: (1) Databricks Workflows email notification to the on-call data engineer; (2) Azure Monitor alert to the PagerDuty on-call rotation for Severity 2 (Gold layer delay) or Severity 1 (streaming fraud pipeline failure); (3) automated Slack message to the #data-platform-ops channel with a direct link to the failed task's log output. The platform maintains a 99.8% pipeline SLA achievement rate over the trailing 12 months, with the 0.2% breaches attributable to upstream source system maintenance windows.

Disaster Recovery

The platform implements an RPO (Recovery Point Objective) of 1 hour and an RTO (Recovery Time Objective) of 4 hours for the Gold layer. ADLS Gen2 geo-redundant storage (GRS) replicates all data to the Azure South Africa West secondary region asynchronously. Databricks cluster configurations and workspace settings are stored in a Terraform repository enabling full workspace re-deployment within 45 minutes. Delta Lake transaction logs enable point-in-time recovery to any hour within the 30-day VACUUM retention window. Monthly DR drills validate the full recovery procedure against the documented RTO/RPO targets.

Infrastructure as Code

All Databricks cluster configurations, workflow definitions, Unity Catalog policies, and ADLS Gen2 lifecycle rules are managed as Terraform code in a GitLab repository. Changes require a pull request review by a senior engineer and automated terraform plan validation before merge. Production deployments are triggered by GitLab CI/CD pipelines with mandatory approval gates.

Chapter 10

Business Intelligence

Eight Power BI Premium dashboards serving every business unit from executive leadership to branch operations.

The consumption tier of the platform delivers insights through eight purpose-built Power BI Premium dashboards, collectively serving 500 active users across the bank. Dashboards are designed against the bank's master data model in the Gold layer, using a mix of DirectQuery (for real-time operational views) and Import mode (for complex analytical views requiring DAX measures). All dashboards are secured through Azure Active Directory row-level security (RLS), ensuring each user sees only the data appropriate to their role and geographic scope.

Figure 10.1 โ€” Fraud Intelligence โ€” 12-Month Trend Analysis

The fraud rate has remained stable at 0.09%โ€“0.14% across 200,000+ monthly card transactions, with the ensemble model's false positive rate declining from 4.1% to 3.2% over the year as the model was recalibrated with more recent confirmed fraud labels. The December seasonal spike in fraud attempts is a well-documented industry pattern driven by increased card transaction volumes during the holiday period โ€” the model's recall performance held stable despite the volume surge, preventing an estimated R45M in fraud losses over the full year.

Left: Monthly fraud rate (%) and false positive rate (%). Right: Monthly cumulative fraud loss prevented (R millions). Source: fact_transaction, silver_card_transactions, June 2025.

Figure 10.2 โ€” NPL Prediction Accuracy: PD Model Forecast vs Actual NPL Ratio (12-Month)

What: This dual-line chart compares the PD model's forward-looking portfolio-level NPL forecast (teal โ€” derived from the monthly batch scoring by aggregating individual customer PD scores to a weighted portfolio average) against the realised NPL ratio recorded in the loan servicing system each month (gold). A shaded ยฑ0.2 percentage-point tolerance band surrounds the predicted line.
Why: The Basel II/III IRB framework requires banks to demonstrate a "use test" โ€” the PD model must be genuinely embedded in business decision-making, not just computed for regulatory reporting. Close predicted-vs-actual alignment proves the model is producing actionable forecasts that management relies on for provisioning, capital allocation, and stress testing. SARB examiners specifically review this chart during model validation inspections. Divergence beyond ยฑ0.5 pp triggers a model recalibration event.
Where: Computed monthly in the Regulatory Reporting notebook (07_regulatory_reporting.py); consumed by the Credit Risk Executive Dashboard and included in the SARB BA210 submission narrative.
How: Batch PD scores from gold.fact_loan_portfolio.pd_score aggregated as exposure-weighted average โ†’ compared to is_npl = true count / total loans. Both series stored in gold.reg_npl_metrics Delta table and served via Power BI DirectQuery.

Teal line = PD model predicted NPL ratio. Gold line = actual NPL ratio from loan servicing. Shaded band = ยฑ0.2 pp tolerance. Max divergence = 0.09 pp (Aug). Calibration RMSE = 0.06 pp โ€” confirms strong predictive accuracy. Source: gold.reg_npl_metrics, FY 2025.

Figure 10.3 โ€” Customer Risk-Value Segmentation Matrix (6 Segments)

What: This bubble chart maps the 500,000-customer base across two strategic dimensions: credit risk (x-axis โ€” bureau credit score, where higher scores indicate lower risk) and customer value (y-axis โ€” average monthly ZAR transaction volume as a proxy for wallet share and revenue potential). Bubble size represents the segment's aggregate ECL provision exposure โ€” larger bubbles require more regulatory capital allocation against that segment.
Why: Pure risk segmentation misses half the picture โ€” a high-risk, high-value customer requires a different strategy than a high-risk, low-value customer. This 2ร—3 matrix enables the bank to operate differentiated strategies per quadrant: Grow (low risk, high value โ†’ cross-sell Premium and Wealth products); Retain (low risk, low value โ†’ digital engagement, fee waivers); Monitor (medium risk, high value โ†’ intensive RM coverage, covenant monitoring); Nurture (medium risk, low value โ†’ financial wellness programmes); Watch (high risk, high value โ†’ credit limit reviews, collateral top-ups); Exit (high risk, low value โ†’ no new credit, collections handoff).
Where: Customer 360 Power BI dashboard (monthly segment refresh); Credit Committee monthly portfolio review; Branch Performance dashboard (filtered by geography).
How: K-means clustering (k=6) on normalised bureau credit score, 90-day transaction RFM (Recency, Frequency, Monetary) features from gold.fact_transaction, and PD score from the ML scoring notebook. Centroids translated to descriptive segment labels by the Risk Analytics team. ECL aggregated per segment from gold.fact_loan_portfolio.ecl_amount_zar.

Bubble chart โ€” x: bureau credit score (higher = lower risk), y: avg monthly transaction volume (ZAR), bubble size: segment ECL provision (Rm). 6 K-means segments. Hover for segment detail. Source: dim_customer + fact_loan_portfolio + ML scoring, June 2025.

Dashboard Inventory

1. Executive Dashboard

Board and C-suite facing overview of the bank's key performance indicators: total assets, net interest income, cost-to-income ratio, customer growth, and capital adequacy. Single-page design with drill-through to business unit detail. Updated daily from Gold layer. Audience: CEO, CFO, CRO, board members.

2. Credit Risk Dashboard

Comprehensive view of the loan portfolio: exposure by product and geography, IFRS 9 stage migration waterfall, PD distribution heatmap, ECL movement analysis, and top-20 large exposures. Includes a Monte Carlo simulation-based ECL sensitivity analysis. Audience: Chief Risk Officer, Credit Committee, Provisioning team.

3. Fraud Intelligence Dashboard

Real-time fraud monitoring with a 5-minute data refresh: fraud score distribution, confirmed fraud by channel and merchant category, fraud rate trend, savings vs. prior year, and a geographic heat map of fraud incidents by postal code. Audience: Fraud Operations, Card Operations, Head of Financial Crime.

4. AML & Financial Crime Dashboard

Anti-money laundering case management view: suspicious transaction volumes, STR pipeline, customer risk rating distribution, high-risk jurisdiction exposure, and PEP (Politically Exposed Person) screening status. Audience: MLRO, Compliance team, FICA officers.

5. Branch Performance Dashboard

Branch-level P&L view for regional managers: deposit book growth, loan origination, fee income, customer satisfaction scores, and staff productivity metrics. Geography-filtered by the user's RLS profile. Audience: Regional Managers, Branch Managers, Retail Banking leadership.

6. Treasury & Liquidity Dashboard

Intraday liquidity position, LCR and NSFR ratios vs. limits, FX exposure by currency, bond portfolio mark-to-market, and funding concentration risk metrics. Refreshed every 30 minutes from treasury source data. Audience: Head of Treasury, ALM Committee, SARB Prudential team.

7. Customer 360 Dashboard

Customer-level analytics accessible to relationship managers: product holding summary, transaction recency/frequency/monetary (RFM) scoring, churn propensity score, cross-sell propensity index, and lifetime value estimate. PII-masked for non-privileged users. Audience: Relationship Managers, Digital Banking, Marketing.

8. Regulatory Reporting Dashboard

Compliance-facing view of all regulatory return status: BA700 submission timeline, IFRS 9 provisioning summary, Basel III capital ratios vs. limits, POPIA compliance score, and FIC STR submission log. Provides a single pane of glass for the Chief Compliance Officer. Audience: CCO, Compliance team, Internal Audit.

Power BI Performance Standards

All dashboards must meet the platform's BI performance SLA: initial page load under 4 seconds, visual refresh under 2 seconds for DirectQuery visuals with applied filters, and paginated report export under 30 seconds. Dashboards failing the SLA threshold are automatically flagged in the Platform Operations dashboard for query optimisation review.

Chapter 11

Data Governance

Unity Catalog, data lineage, access controls, PII masking, and data retention policies.

Data governance is not a layer on top of the platform โ€” it is embedded within it. Unity Catalog provides the centralised governance control plane, while the Medallion architecture's clear layer boundaries enforce the governance model structurally. Every data asset on the platform โ€” tables, views, ML models, dashboards โ€” is catalogued, classified, owned, and access-controlled through a unified framework aligned with the bank's Enterprise Data Management Policy.

Unity Catalog Structure

-- Unity Catalog hierarchy prime_capital_bank (metastore) โ”œโ”€โ”€ bronze_catalog โ”‚ โ”œโ”€โ”€ core_banking -- Raw ingested tables from T24 โ”‚ โ”œโ”€โ”€ crm -- Salesforce raw data โ”‚ โ”œโ”€โ”€ cards -- Card system tables โ”‚ โ””โ”€โ”€ external -- Third-party feeds โ”œโ”€โ”€ silver_catalog โ”‚ โ”œโ”€โ”€ customers -- Cleansed, SCD2 customer entities โ”‚ โ”œโ”€โ”€ accounts -- Account master, enriched โ”‚ โ”œโ”€โ”€ transactions -- Standardised transaction history โ”‚ โ””โ”€โ”€ reference -- Canonical reference data โ””โ”€โ”€ gold_catalog โ”œโ”€โ”€ dimensions -- Star schema dimension tables โ”œโ”€โ”€ facts -- High-grain fact tables โ”œโ”€โ”€ regulatory -- Compliance reporting marts โ””โ”€โ”€ ml_features -- Feature Store tables

Access Control Model

RoleBronze AccessSilver AccessGold AccessPII Visible
Data EngineerRead/WriteRead/WriteRead/WriteNo (masked)
Data ScientistRead (selected)ReadReadNo (masked)
BI AnalystNoneNoneReadNo (masked)
Risk AnalystNoneRead (risk)ReadNo (masked)
Compliance OfficerNoneRead (AML)ReadYes (with MFA)
PII Reader (privileged)NoneReadReadYes
Auditor (read-only)Read (audit)ReadReadNo

Data Lineage

Unity Catalog automatically captures table-level and column-level lineage for all Spark SQL and dbt transformations executed within the Databricks workspace. Lineage is queryable via the Unity Catalog API and visualised in the Databricks UI โ€” showing, for example, that the gold.facts.fact_loan_portfolio.ecl_provision_zar column derives from the PD model feature, which in turn derives from the credit bureau bronze table. This end-to-end lineage satisfies BCBS 239 (Risk Data Aggregation and Risk Reporting) principles and provides the audit trail required for SARB regulatory inspections.

Data Retention & Lifecycle

ADLS Gen2 lifecycle management policies automate tier transitions: data older than 90 days in the Bronze and Silver layers moves from Hot to Cool tier (60% cost saving); data older than 2 years moves to Archive tier (90% cost saving). Gold layer data remains on Hot tier for 5 years, aligning with the bank's Records Retention Schedule and POPIA's storage limitation principle. Delta Lake table history is retained for 30 days for operational recovery, with regulatory snapshots committed to Archive storage for 7-year statutory retention (Companies Act 71 of 2008).

POPIA Compliance Implementation

The platform enforces POPIA's eight conditions for lawful processing through: purpose limitation (Unity Catalog tags define permitted processing purposes per table); storage limitation (automated ADLS lifecycle policies); data subject rights (automated DSAR response pipeline identifying all PII instances for a given customer); and breach notification (Azure Defender for Cloud alerts triggering the 72-hour POPIA breach notification workflow).

Chapter 12

SA Fintech Ecosystem & Cards

Prime Capital Bank's integration with South Africa's leading fintech payment partners โ€” Yoco, SnapScan, PayFast, Peach Payments, and PayGate โ€” and the data model powering settlement analytics.

South Africa's payment technology landscape has undergone a structural transformation since 2012. The emergence of app-based QR payments, SME-focused card acquiring, and developer-first online gateways has created a fragmented but dynamic ecosystem sitting alongside the traditional card rails (Visa, Mastercard) and bank-owned channels (EFT, RTC). Prime Capital Bank has positioned itself as the settlement backbone for this ecosystem โ€” providing acquiring bank infrastructure, settlement accounts, and risk oversight for five key fintech partners whose combined annual processing volume exceeds R27.8 billion.

Strategic Position

PCB acts as the acquiring bank for Yoco and SnapScan card transactions, and as the settlement bank for PayFast, Peach Payments, and PayGate online gateway flows. This gives the bank first-mover visibility into SME cash flow data โ€” a powerful input to credit scoring models for the underbanked merchant segment.

Partner Comparison

PartnerPlatform TypeMDRMin FeeAvg BasketSettlementMerchant BaseAnnual Volume
YocoPOS / Card Acquiring2.95%โ€”R856T+1250,000+R12.4B
SnapScanQR Code Payment2.75%โ€”R340T+180,000+R3.2B
PayFastOnline Gateway3.50%R2.00R1,251T+245,000+R8.1B
Peach PaymentsOnline Gateway2.90%R1.50R980T+112,000+R2.3B
PayGateOnline Gateway2.80%R1.00R1,891T+28,000+R1.8B

Partner Profiles

Yoco โ€” South Africa's SME Payment Champion

Founded in Cape Town in 2013, Yoco has grown to become South Africa's largest independent payment provider with over 250,000 active SME merchants. The product line spans four card machine tiers โ€” Yoco Go (R599), Yoco Khumo (R699), Yoco Neo (R1,499), and Yoco Counter + Neo Touch (R2,999) โ€” serving the full spectrum from market traders to restaurant chains. Following its USD 83 million Series C raise in 2021 (led by Dragoneer, with existing investors Partech and Quona), Yoco has expanded into business banking services including working capital loans, business insurance, and a business management app. The 2.95% MDR is among the most competitive in the POS category and has been the primary driver of merchant adoption versus traditional bank card machines at 3.5โ€“5.0%.

SnapScan โ€” Standard Bank's QR Payment Arm

SnapScan pioneered QR-code-based payments in South Africa after its founding in 2012, and was acquired by Standard Bank in 2014. The platform requires no card machine โ€” merchants display a static QR code and customers pay via the SnapScan app. This zero-hardware model dramatically lowered the barrier to entry for informal and pop-up merchants: food trucks, market vendors, street food traders, and event operators. The 2.75% MDR and R340 average basket reflect the hospitality and food-and-beverage concentration of its merchant base. As a Standard Bank subsidiary, SnapScan is a direct competitor of PCB in the SME merchant acquiring space, making PCB's settlement bank relationship strategically important for margin defence.

PayFast โ€” South Africa's E-Commerce Gateway

Founded in Cape Town in 2007, PayFast is the dominant online payment gateway for South African e-commerce with integrations across Shopify, WooCommerce, Magento, Wix, and PrestaShop. Its 20+ supported payment methods โ€” including credit/debit cards, instant EFT, Mobicred, Zapper, and SiD Secure EFT โ€” made it the default choice for SA merchants going online. Acquired by DPO Group in 2021 (which was itself subsequently acquired by Network International in 2022), PayFast now forms part of Africa's largest payment processing group. The 3.50% + R2.00 fee structure is the highest in the comparison set, justified by the complexity of multi-method checkout, fraud screening, and the broad integration ecosystem. The R1,251 average basket reflects the higher-value e-commerce purchase context.

Peach Payments โ€” The Developer-First Gateway

Peach Payments was founded in Cape Town in 2012 with a deliberate focus on API-first architecture and developer experience. Its REST API, hosted payment page, and tokenisation library are widely regarded as the most technically sophisticated in the SA market. Merchants include Superbalist, OneDayOnly, and major hospitality chains. The 2.90% + R1.50 fee structure is the most competitive among online gateways, reflecting a deliberate strategy to win mid-to-large merchants on economics. Peach Payments supports 3D Secure 2.0, recurring billing, and subscription management โ€” capabilities driving rapid growth in the B2B SaaS and subscription commerce verticals. Settlement on T+1 versus competitors' T+2 provides a working capital advantage to merchants.

PayGate โ€” The Enterprise Incumbent

Established in Johannesburg in 2000, PayGate is one of the oldest SA online payment gateways and serves enterprise-tier merchants including large national retailers, travel agencies, and government e-services platforms. Also a DPO Group subsidiary (Network International), PayGate occupies the high-basket end of the market with a R1,891 average transaction value โ€” reflecting the enterprise procurement, B2B invoice, and travel booking use cases. While its technology stack predates the API-first era, PayGate's strength lies in its deep enterprise integrations, recurring billing infrastructure, and 25+ years of institutional trust. PCB's settlement relationship with PayGate covers the largest average transaction values in the fintech portfolio.

Annual Settlement Volume by Partner

2024 estimated annual gross settlement volumes (R billions). Source: PCB Acquiring System โ€” FACT_FINTECH_SETTLEMENT aggregated.

Monthly Settlement Trend โ€” 2024

Combined monthly gross volume across all five partners (R millions). Black Friday (November) and Festive Season (December) peaks clearly visible.

SA Payment Ecosystem โ€” Market Context

The total South African card and digital payment market processes approximately R1.8 trillion per year across all channels. The five fintech partners integrated with PCB represent roughly 1.5% of this market by value but a disproportionately high share of SME merchant transaction count โ€” the fastest-growing segment by number of merchants. Key macro drivers:

Data Model: Cards & Fintech Domain

Two new Gold layer tables underpin the fintech analytics capability:

-- DIM_FINTECH: Reference dimension (5 rows โ€” one per partner) prime_capital.gold.dim_fintech โ”œโ”€โ”€ fintech_sk BIGINT surrogate key โ”œโ”€โ”€ fintech_id STRING FT001 โ€“ FT005 โ”œโ”€โ”€ fintech_name STRING Yoco / SnapScan / PayFast / Peach / PayGate โ”œโ”€โ”€ platform_type STRING POS Card Acquiring / QR / Online Gateway โ”œโ”€โ”€ fee_rate_pct DECIMAL MDR as a percentage (2.75 โ€“ 3.50) โ”œโ”€โ”€ min_fee_zar DECIMAL Per-transaction minimum (0.00 โ€“ 2.00) โ”œโ”€โ”€ target_segment STRING SME / E-Commerce / Enterprise โ”œโ”€โ”€ active_merchants_est INT Live merchant count โ”œโ”€โ”€ annual_volume_zar_bn DECIMAL Estimated annual volume (R billions) โ””โ”€โ”€ settlement_days INT T+N settlement cycle -- FACT_FINTECH_SETTLEMENT: Monthly grain (60 rows โ€” 12 months ร— 5 partners) prime_capital.gold.fact_fintech_settlement โ”œโ”€โ”€ settlement_sk BIGINT surrogate key โ”œโ”€โ”€ fintech_sk BIGINT FK โ†’ dim_fintech โ”œโ”€โ”€ settlement_month DATE Month-end date (partition key) โ”œโ”€โ”€ total_transactions BIGINT Transaction count โ”œโ”€โ”€ gross_volume_zar DECIMAL Gross merchant volume settled โ”œโ”€โ”€ merchant_fees_zar DECIMAL Fee revenue (gross_volume ร— MDR) โ”œโ”€โ”€ interchange_fees_zar DECIMAL Fees paid to card networks โ”œโ”€โ”€ net_settlement_zar DECIMAL Amount settled to merchants โ”œโ”€โ”€ avg_transaction_zar DECIMAL Average basket value โ”œโ”€โ”€ chargebacks_count INT Chargeback volume โ”œโ”€โ”€ chargeback_ratio_pct DECIMAL Chargeback rate (%) โ”œโ”€โ”€ failed_transactions INT Declines and failures โ”œโ”€โ”€ failure_rate_pct DECIMAL Failure rate (%) โ””โ”€โ”€ new_merchants_onboarded INT Net new merchant additions

Chargeback Ratio by Partner

Average annual chargeback ratio (%) by partner. POS/QR channels show lower rates than online gateways due to card-present authentication and biometric verification. Industry benchmark: <0.20%.

Integration Architecture

The fintech settlement data flows into PCB's data platform through a dedicated acquiring integration pipeline:

Fintech Partner Acquiring System โ””โ”€ Daily Settlement Report (CSV / SFTP / API) โ””โ”€ Azure Data Factory โ€” Bronze ingestion โ””โ”€ bronze.cards.fintech_settlement_raw โ””โ”€ Silver cleanse & validate โ””โ”€ silver.cards.fintech_settlement โ””โ”€ Gold aggregation (monthly grain) โ””โ”€ gold.fact_fintech_settlement โ”œโ”€ Power BI: Cards & Fintech Dashboard โ”œโ”€ ML Feature Store: merchant_risk_features โ””โ”€ Regulatory: Card Scheme Settlement Report
Business Value: SME Credit Scoring

The monthly gross_volume_zar and new_merchants_onboarded signals from FACT_FINTECH_SETTLEMENT feed directly into PCB's SME Credit Risk ML model. Merchants with 6+ months of consistent settlement history and sub-0.15% chargeback ratios qualify for pre-approved business credit lines โ€” reducing manual underwriting time from 5 days to under 4 hours and expanding credit access to the underbanked SME segment.

Chapter 13

Appendix

Glossary of banking and data terms, table catalog reference, and contact information.

Glossary of Terms

TermDefinition
AMLAnti-Money Laundering โ€” regulatory framework for detecting and preventing money laundering activities.
Acquiring BankThe bank that processes card payments on behalf of merchants. PCB acts as acquiring bank for Yoco and SnapScan transactions, giving it first-party data on SME cash flows.
AUC-ROCArea Under the Receiver Operating Characteristic curve โ€” primary performance metric for classification ML models.
BA700SARB's standard bank return for monthly prudential data submission by registered banks.
Basel IIIInternational regulatory framework for bank capital adequacy, stress testing, and liquidity risk, implemented by SARB.
CET1Common Equity Tier 1 โ€” the highest quality regulatory capital, consisting of ordinary shares and retained earnings.
Delta LakeOpen-source ACID-compliant storage layer on top of Parquet, providing time-travel, schema enforcement, and MERGE operations.
dbtData Build Tool โ€” SQL-first transformation framework providing modular, tested, documented data models.
EADExposure at Default โ€” estimate of the outstanding loan balance at the time a borrower defaults.
ECLExpected Credit Loss โ€” IFRS 9 provision methodology replacing the IAS 39 incurred-loss model.
EFTElectronic Funds Transfer โ€” the primary interbank payment rail in South Africa, operated by BankservAfrica.
FICAFinancial Intelligence Centre Act โ€” South African AML/KYC legislation governing customer due diligence and STR reporting.
IFRS 9International Financial Reporting Standard 9 โ€” the accounting standard governing classification, measurement, and impairment of financial instruments.
LCRLiquidity Coverage Ratio โ€” Basel III metric requiring banks to hold sufficient high-quality liquid assets to survive a 30-day stress scenario.
LGDLoss Given Default โ€” estimate of the percentage of EAD the bank will lose if a borrower defaults.
MDRMerchant Discount Rate โ€” the percentage fee charged to a merchant on each payment transaction. PCB's fintech partners charge between 2.75% (SnapScan) and 3.50% (PayFast).
Medallion ArchitectureA data design pattern organising data in Bronze (raw), Silver (clean), and Gold (analytical) quality tiers.
MLflowOpen-source ML lifecycle platform for experiment tracking, model packaging, and model registry.
NSFRNet Stable Funding Ratio โ€” Basel III metric requiring banks to maintain stable funding relative to assets over a 1-year horizon.
PDProbability of Default โ€” the likelihood that a borrower will fail to meet their debt obligations within a defined time horizon.
POPIAProtection of Personal Information Act โ€” South Africa's primary data privacy legislation, effective July 2021.
RTCReal-Time Clearing โ€” South Africa's domestic immediate payment scheme, settled via SARB SAMOS.
RWARisk-Weighted Assets โ€” the denominator in capital adequacy calculations, weighting assets by their credit risk.
SARBSouth African Reserve Bank โ€” the central bank and prudential regulator of South African banks.
SCD2Slowly Changing Dimension Type 2 โ€” a dimension table design that preserves historical attribute values with effective and expiry dates.
SIFISystemically Important Financial Institution โ€” a bank whose failure could pose systemic risk to the financial system.
STRSuspicious Transaction Report โ€” a mandatory report submitted to the Financial Intelligence Centre under FICA.
PayShapSouth Africa's real-time low-value payment scheme launched in 2023 under SARB's Rapid Payments Programme (RPP). Competes with fintech QR payment channels for instant consumer payments.
Unity CatalogDatabricks' centralised data governance solution providing unified metastore, access control, lineage, and auditing across workspaces.
YocoSouth Africa's largest independent POS card acquiring platform, serving 250,000+ SME merchants. PCB is the acquiring bank for Yoco transactions, settling at T+1 at 2.95% MDR.

Full Table Catalog

A complete table-level reference is maintained in the separate Data Dictionary document (docs/data_dictionary/data_dictionary.md), which provides column-level documentation, business rules, and PII classification for all 247 catalogued tables across the Bronze, Silver, and Gold layers.

Contact Information

RoleTeamResponsibility
Chief Data OfficerData StrategyPlatform vision and data governance policy
Head of Data EngineeringData EngineeringPlatform architecture and pipeline ownership
Lead Data ScientistAnalytics & AIML model development and governance
Data Governance ManagerData GovernanceUnity Catalog, PII policy, POPIA compliance
Platform SRE LeadSite ReliabilityPipeline SLA, incident response, DR
Regulatory Reporting LeadFinance & ComplianceSARB/IFRS reporting, BA700, Basel III
Document Control

This document is classified CONFIDENTIAL โ€” INTERNAL USE ONLY. Version 3.1, approved June 2025. Review cycle: quarterly. Owner: Head of Data Engineering, Prime Capital Bank Limited. Unauthorised distribution is prohibited under the bank's Information Security Policy and POPIA Section 19 obligations.

— End of Document —

Prime Capital Bank Limited  ·  Registration No. 1968/000123/06  ·  Authorised Financial Services Provider (FSP 12345)  ·  Registered Credit Provider (NCRCP 9876)
Prudential Authority Licence No. PA12345  ·  ยฉ 2025 Prime Capital Bank Limited. All rights reserved.