uchicago-data-analytics-data-quality-profiling

Data Quality Profiling

Purpose

Prevent users from trusting weak data too quickly. Profile quality before insights when data is new, messy, surprising, or used for decisions.

Required Checks

For each dataset or query result, assess:

• row count and column count
• grain: one row per what
• candidate keys and duplicate risk
• missing values in important fields
• mixed data types and invalid dates/numbers
• unexpected categories or placeholder values
• outliers and impossible values
• freshness or extract date when available
• formula errors, hidden sheets, stale pivots for Excel
• sensitive fields requiring aggregation or exclusion

Output

Return a concise readiness summary:

Data readiness: high / medium / low
Main risks:
Recommended fixes before analysis:
Safe analyses now:
Analyses to avoid until fixed:

Do not block every analysis. If risks are minor, state assumptions and proceed. If risks can change the answer, pause and ask the user whether to clean, filter, or continue with caveats.

Save Point

After profiling a substantial file or dataset, offer to save/update DATASETS.md using uchicago-data-analytics-analysis-state.