Exercises
Every exercise belongs to a lesson and comes with a Python notebook you can run in Google Colab without installing anything, and a worked solution notebook to check your work against. Data is always synthetic or openly licensed and de-identified; datasets that need credentialed access are linked with instructions, never copied.
Foundations
- Healthcare data types: records, images, signals, notes and omics
Generate synthetic patient records with Synthea (or use its sample data), load them with pandas, and build a one-row-per-patient table with explicit choices about time windows and missing values.
Beginner · Clinical · Foundations · Data: Synthea synthetic patient records (CSV export)
- Data standards: FHIR, HL7, ICD, SNOMED CT, LOINC and OMOP
Read synthetic FHIR R4 patient bundles from Synthea, extract HbA1c and glucose Observations by LOINC code, and harmonise their units into one analysis table.
Beginner · Technical · Foundations · Data: Synthea synthetic FHIR R4 bundles
- Privacy and de-identification: HIPAA, GDPR and India's DPDP Act
Measure and reduce re-identification risk in a synthetic patient table, and see what each step costs in analytical detail.
Beginner · Clinical · Foundations · Data: Synthetic patients generated in the notebook (no real patient data)
- Setting up Python and Colab for clinical data work
Set up a working Python environment, then load, check, summarise and plot a small synthetic clinical table, handling missing values explicitly.
Beginner · Technical · Foundations · Data: Synthetic table written into the notebook (no real patient data)
- Diagnostic accuracy metrics: sensitivity, specificity, PPV, ROC and calibration
Explore how threshold and prevalence change the metrics of a simulated risk model, and check its calibration.
Beginner · Clinical · Foundations · Data: Synthetic patients generated in the notebook (no real patient data)
Machine learning
- Prediction or causation? Framing the clinical question
Explore confounding by indication in a simulated cohort, and see when adjustment does and does not recover the true treatment effect.
Beginner · Clinical · Machine learning · Data: Synthetic patients generated in the notebook (no real patient data)
- Building a cohort and avoiding data leakage
Build a 30-day readmission cohort from Synthea data with an explicit time zero, then measure how much each kind of leakage inflates performance.
Intermediate · Technical · Machine learning · Data: Synthea synthetic encounters and observations (CSV export)
- A logistic-regression risk score
Develop, evaluate and simplify a logistic-regression risk model on synthetic data, and test how it holds up when the population changes.
Intermediate · Technical · Machine learning · Data: Synthetic adults generated in the notebook (no real patient data)
- Gradient-boosted trees for readmission risk
Compare gradient boosting with logistic regression for 30-day readmission, fairly and on several metrics.
Intermediate · Technical · Machine learning · Data: Synthetic discharges generated in the notebook (no real patient data)
- Survival analysis: time-to-event models
Analyse a synthetic kidney-failure cohort with survival methods, and see how censoring and competing risks change the answer.
Intermediate · Technical · Machine learning · Data: Synthetic cohort generated in the notebook (no real patient data)
- Missing data and class imbalance
Handle informative missingness and a rare outcome properly, and measure the consequences of common shortcuts.
Intermediate · Technical · Machine learning · Data: Synthetic patients generated in the notebook (no real patient data)
- Explaining models with SHAP
Use SHAP to explain a readmission model, find its interaction, and test how explanations change with correlated inputs.
Intermediate · Technical · Machine learning · Data: Synthetic discharges generated in the notebook (no real patient data)
- Calibration and decision-curve analysis
Detect miscalibration at a new site, recalibrate, and judge clinical usefulness with a decision curve.
Intermediate · Clinical · Machine learning · Data: Synthetic sites generated in the notebook (no real patient data)
- Unsupervised learning for patient phenotypes
Cluster synthetic diabetes profiles, test whether the clusters are real and stable, and compare them with continuous variables for predicting an outcome.
Intermediate · Technical · Machine learning · Data: Synthetic profiles generated in the notebook (no real patient data)
Deep learning & imaging
- Neural-network basics
Build, train and probe a small neural network, and see when it helps over logistic regression.
Intermediate · Technical · Deep learning & imaging · Data: Synthetic data generated in the notebook (scikit-learn make_moons)
- CNNs for chest X-ray and retinal images
Train CNNs on synthetic and real medical images, and test them for shortcut learning.
Intermediate · Technical · Deep learning & imaging · Data: Synthetic scans in the notebook; MedMNIST PneumoniaMNIST and RetinaMNIST (optional download)
- Image segmentation with U-Net
Train and evaluate a U-Net, improve it for small lesions, and report results responsibly.
Advanced · Technical · Deep learning & imaging · Data: Synthetic scans and masks in the notebook; optional open segmentation dataset under its licence
- ECG and physiological time-series models
Compare feature-based and deep-learning rhythm classifiers on synthetic and real ECGs.
Intermediate · Technical · Deep learning & imaging · Data: Synthetic strips in the notebook; PTB-XL 12-lead ECG dataset from PhysioNet (optional)
- Transfer learning and imaging foundation models
Compare transfer-learning strategies on synthetic and real images, and plan the validation of a pretrained model.
Advanced · Technical · Deep learning & imaging · Data: Synthetic scans in the notebook; optional MedMNIST RetinaMNIST or PneumoniaMNIST; optional torchvision pretrained ResNet-18
Clinical NLP
- Working with clinical notes
Build a small clinical-text preprocessing pipeline and test where rules break.
Beginner · Technical · Clinical NLP · Data: Twenty synthetic clinic notes written for the notebook
- Named-entity recognition and clinical coding
Build, score and improve a clinical entity extractor on synthetic notes.
Intermediate · Technical · Clinical NLP · Data: Thirty synthetic notes with coder-style labels, written for the notebook
- Clinical text embeddings
Build and stress-test embedding-based note classifiers.
Intermediate · Technical · Clinical NLP · Data: Synthetic notes generated in the notebook
- De-identifying clinical free text
Build, evaluate and improve a de-identifier on synthetic notes.
Intermediate · Technical · Clinical NLP · Data: Fifteen synthetic notes with marked identifiers, written for the notebook
Generative AI & LLMs
- How large language models work
Explore how next-token sampling and temperature create variability and plausible errors, using toy models that run without any API.
Beginner · Clinical · Generative AI & LLMs · Data: Synthetic sentences written in the notebook (no real patient data)
- Prompting for clinical tasks
Design, validate and evaluate a prompt that extracts diabetes information from synthetic clinic notes.
Intermediate · Technical · Generative AI & LLMs · Data: Twenty synthetic clinic notes with clinician-style labels, written for the notebook
- Medical LLMs and their benchmarks
Analyse benchmark results like a reviewer: quantify uncertainty, compare models properly and look for warning signs.
Intermediate · Clinical · Generative AI & LLMs · Data: Simulated benchmark answers generated in the notebook
- Hallucination and grounding
Measure and reduce hallucinations and omissions in synthetic AI-generated clinic notes.
Intermediate · Clinical · Generative AI & LLMs · Data: Ten synthetic consultation transcripts with AI-style draft notes, written for the notebook
- Fine-tuning with LoRA on synthetic data
Implement and explore LoRA in NumPy, then plan a responsible clinical fine-tuning project.
Advanced · Technical · Generative AI & LLMs · Data: Synthetic matrices and examples generated in the notebook
- Multimodal models: text, images and more
Explore zero-shot classification and shortcut learning with toy embeddings, then with an open chest X-ray dataset if available.
Intermediate · Technical · Generative AI & LLMs · Data: Toy embeddings generated in the notebook; optional extension uses an openly licensed chest X-ray dataset
- Summarisation and ambient clinical documentation
Audit synthetic AI-drafted notes against their final versions and design a monitoring report.
Beginner · Clinical · Generative AI & LLMs · Data: Twenty synthetic draft and final note pairs, written for the notebook
Retrieval-augmented generation (RAG)
- Why RAG? Grounding answers in trusted sources
Build a minimal RAG pipeline over a synthetic document library and observe where it succeeds and fails.
Beginner · Clinical · Retrieval-augmented generation (RAG) · Data: Synthetic protocol documents written for the notebook
- Chunking clinical guidelines
Compare chunking strategies on the synthetic protocol and measure which keeps clinical rules intact.
Intermediate · Technical · Retrieval-augmented generation (RAG) · Data: Synthetic protocol in the notebook; optional openly licensed guideline excerpt (for example WHO publications under CC BY-NC-SA 3.0 IGO)
- Embeddings and vector stores
Compare keyword and embedding retrieval, improve abbreviation handling and build a filtered vector store.
Intermediate · Technical · Retrieval-augmented generation (RAG) · Data: Synthetic protocol in the notebook; small open embedding model all-MiniLM-L6-v2
- Evaluating retrieval
Build and use a retrieval test set to choose and improve a retrieval method.
Intermediate · Technical · Retrieval-augmented generation (RAG) · Data: Synthetic protocol and labelled questions in the notebook
- Citations and answer grounding
Build and stress-test a claim-level citation checker on the synthetic protocol.
Intermediate · Technical · Retrieval-augmented generation (RAG) · Data: Synthetic protocol and answer claims in the notebook
- Build a guideline Q&A assistant
Build, test and improve a guideline Q&A assistant over the synthetic protocol.
Intermediate · Technical · Retrieval-augmented generation (RAG) · Data: Synthetic protocol in the notebook; small open embedding model all-MiniLM-L6-v2
- End-to-end RAG evaluation
Evaluate and improve the guideline assistant end to end, and design a realistic evaluation plan.
Intermediate · Technical · Retrieval-augmented generation (RAG) · Data: Synthetic protocol and labelled questions in the notebook
Agentic AI
- Agents and tool use
Build a small tool-using agent loop, add validation and safeguards, and test how it handles bad inputs.
Intermediate · Technical · Agentic AI · Data: Synthetic cases written in the notebook (no real patient data)
- Tool calling against FHIR data
Build and test patient-scoped FHIR tools over synthetic data, then connect them to the agent loop.
Intermediate · Technical · Agentic AI · Data: Synthetic FHIR bundle in the notebook; optional Synthea FHIR R4 bundles
- Multi-agent clinical workflows
Build a small multi-agent pipeline over synthetic FHIR data, inject errors and measure how well independent checks catch them.
Intermediate · Clinical · Agentic AI · Data: Synthetic FHIR bundle from the previous lesson
- MCP and tool servers
Build a small read-only MCP server over synthetic data, connect a client, and add defences against poisoned or changed tools.
Advanced · Technical · Agentic AI · Data: Synthetic observations in the notebook
- Guardrails and human-in-the-loop design
Build layered guardrails around a scripted clinical agent and test them with adversarial inputs.
Intermediate · Clinical · Agentic AI · Data: Synthetic messages, notes and tool calls in the notebook
- Evaluating agents
Build a small trajectory-based evaluation suite for a scripted clinical agent and measure its reliability.
Advanced · Technical · Agentic AI · Data: Synthetic FHIR bundle and tasks written in the notebook
Evaluation, safety & deployment
- External validation and dataset shift
Explore how different kinds of dataset shift affect a clinical prediction model, and fix what can be fixed.
Intermediate · Technical · Evaluation, safety & deployment · Data: Synthetic sites generated in the notebook (no real patient data)
- Bias and fairness
Audit a synthetic model for label bias and subgroup performance, and explore fairness trade-offs.
Intermediate · Clinical · Evaluation, safety & deployment · Data: Synthetic populations generated in the notebook (no real patient data)
- Reporting guidelines: TRIPOD+AI, CONSORT-AI, SPIRIT-AI, DECIDE-AI, CLAIM and more
Appraise a fictional clinical AI paper against the right reporting guideline and PROBAST+AI.
Beginner · Clinical · Evaluation, safety & deployment · Data: A fictional study summary written for the notebook
- Red-teaming LLM applications
Build, run and extend a red-team suite for a synthetic diabetes-advice assistant.
Intermediate · Clinical · Evaluation, safety & deployment · Data: Synthetic red-team prompts written in the notebook
- Monitoring and MLOps
Build a monitoring dashboard for a synthetic deployed model and design the response to drift.
Intermediate · Technical · Evaluation, safety & deployment · Data: Synthetic monthly data generated in the notebook (no real patient data)
Regulation, ethics & governance
- Software as a Medical Device (SaMD)
Practise deciding whether software is a medical device and categorising its risk.
Beginner · Clinical · Regulation, ethics & governance · Data: Ten fictional product descriptions written for the notebook
- US FDA: AI-enabled devices and change-control plans
Read FDA device records critically and apply the CDS criteria and PCCP concepts.
Intermediate · Clinical · Regulation, ethics & governance · Data: The FDA's public AI-enabled device list (download) and fictional product descriptions
- The EU AI Act and medical devices
Classify fictional medical AI products under MDR Rule 11 and identify their AI Act obligations.
Intermediate · Clinical · Regulation, ethics & governance · Data: Fictional product descriptions written for the notebook
- UK MHRA: AI as a medical device
Map what a fictional AI product needs before it can be used in an NHS trust.
Intermediate · Clinical · Regulation, ethics & governance · Data: A fictional AI product description written for the notebook
- India: CDSCO and medical-device software
Work out the Indian regulatory path for fictional AI tools and the ethical review they would need.
Intermediate · Clinical · Regulation, ethics & governance · Data: Fictional product descriptions written for the notebook
- WHO guidance on AI ethics and large multimodal models
Apply WHO's principles to a real-world style AI deployment decision.
Beginner · Clinical · Regulation, ethics & governance · Data: A fictional deployment proposal written for the notebook
- Liability and consent
Analyse responsibility and disclosure for fictional AI-related incidents.
Beginner · Clinical · Regulation, ethics & governance · Data: Three fictional incident scenarios written for the notebook
Applied case studies: diabetes & metabolic health
- Forecasting glucose from CGM data
Forecast glucose from CGM data and evaluate forecasts by their clinical usefulness.
Intermediate · Technical · Applied case studies: diabetes & metabolic health · Data: Synthetic CGM traces generated in the notebook; optional simglucose (open-source UVA/Padova 2008 implementation) or OhioT1DM under its data use agreement
- Diabetic retinopathy screening
Plan the operating point and monitoring for an AI retinopathy screening programme.
Intermediate · Clinical · Applied case studies: diabetes & metabolic health · Data: Synthetic screening population in the notebook; optional open retinal image datasets (for example MedMNIST RetinaMNIST)
- Diabetic kidney disease risk modelling
Model kidney risk in diabetes with categories, continuous predictors and survival methods.
Intermediate · Clinical · Applied case studies: diabetes & metabolic health · Data: Synthetic cohort generated in the notebook (no real patient data)
- Diabetic foot-ulcer image classification
Build and audit a foot-ulcer image classifier across skin tones, and design a healing-monitoring rule.
Intermediate · Clinical · Applied case studies: diabetes & metabolic health · Data: Synthetic wound photos in the notebook; optional DFUC datasets under their data-use terms
- A RAG assistant over a diabetes guideline
Extend the capstone assistant and prepare it for a governance review.
Advanced · Technical · Applied case studies: diabetes & metabolic health · Data: Synthetic protocol and test questions from the course
Educational content only. This material teaches methods; it is not clinical advice. Code, models and examples here are not validated medical devices and must not be used to make decisions about real patients without proper clinical validation, governance and any required regulatory approval. Examples use synthetic or openly licensed de-identified data only. Never paste identifiable patient information into public AI tools. Mentions of models, tools or vendors are for identification, not endorsement.