Intelligent Health Data
Harmonization

Transform chaotic raw health data into structured, analysis-ready formats with complete reproducibility, transparency, and AI-assisted intelligence. Built for national-scale datasets and trusted by leading scientific publications.

100%
Reproducible
AI-Powered
Mapping
Production
Ready
Get Started View on GitHub
config.yaml
mapping:
  source_column: "icd10_code"
  sources:
    - type: "ai"
      threshold: 0.85

# AI-assisted harmonization
# with human-in-the-loop

Enterprise-Grade Features

Built for researchers, ministries of health, and academic institutions worldwide

🔬

100% Reproducible

Every transformation is logged. Run the same config with the same data, get identical results. Perfect for peer review and scientific publication.

📊

Publication-Ready Reports

Automatically generates methodological appendix reports in markdown format. Complete audit trail for every decision made during harmonization.

🤖

AI-Assisted Mapping

Transformer models suggest GBD cause mappings with confidence scores. Human-in-the-loop ensures trust and accuracy.

⚡

Configuration-Driven

Single YAML file controls the entire pipeline. No code changes needed. Non-technical users can configure complex workflows.

🔍

Data Quality Assurance

Comprehensive quality checks with customizable validation rules. Quality scores and issue reporting built-in.

🧩

Extensible Architecture

Plugin system for custom data sources, mapping targets, and quality checks. Designed for community contributions.

Robust Architecture

Modular, tested, and designed for scale

📥

I/O Handlers

CSV, Excel, Parquet support with plugin architecture

→
🧹

Data Cleaning

8 built-in rules with configurable parameters

→
🔗

Mapping Engine

Direct, fuzzy, and AI-powered mapping

→
✅

Quality Checks

9 validation checks with scoring

→
📄

Reporting

Publication-ready markdown reports

Technology Stack

Python 3.10+ Modern Python with type hints
Pandas Data manipulation & analysis
Pydantic Configuration validation
RapidFuzz Fuzzy string matching
Sentence Transformers AI-powered embeddings
Pytest Comprehensive test suite

Visual Configuration Builder

Create config files without writing YAML

Streamlit Web Interface

Our visual config builder makes it easy for non-technical users to create complex configurations:

  • ✨ Visual form-based editor
  • 👀 Live YAML preview
  • ✓ Real-time validation
  • 📥 Download config files
  • 📤 Upload and edit existing configs
# Launch the config builder
streamlit run autogbd/app.py

# Or use the CLI
autogbd config-builder
AutoGBD Config Builder
📝 Editor
👀 Preview
ℹ️ About
Remove Duplicates
Normalize Sex Values

Get Started in Minutes

Installation and quick start guide

Terminal 
# Install AutoGBD
pip install autogbd[app]

# Or install from source
git clone https://github.com/m-aljasem/autogbd.git
cd autogbd
pip install -e ".[app]"
config.yaml 
io:
  input_file: "data/raw_data.csv"
  output_file: "output/harmonized.csv"

mapping:
  source_column: "icd10_code"
  sources:
    - type: "ai"
      threshold: 0.85

quality:
  enabled: true
  checks: []
Terminal 
# Run the harmonization pipeline
autogbd run --config config.yaml

# Validate configuration first
autogbd validate --config config.yaml

# Launch visual config builder
autogbd config-builder

Trusted by Researchers Worldwide

Production-ready for real-world applications

🏥

Ministries of Health

Harmonize national mortality data for GBD studies. Handle millions of records with complete audit trails for regulatory compliance.

🎓

Academic Research

Reproducible data pipelines for peer-reviewed publications. Complete methodological transparency meets journal requirements.

🌍

Multi-Country Studies

Harmonize data from diverse sources across countries. Support for multiple ICD versions and local coding systems.

Comprehensive Documentation

Everything you need to succeed

📖

Full Documentation

Complete API reference, configuration guide, and examples
🚀

Quick Start Guide

Get up and running in minutes with step-by-step instructions
💡

Examples

Real-world examples and configuration templates
🤝

Contributing

Join our community and contribute to the project