CausalArena leaderboard · public release

One arena, many causal discovery regimes.

CausalArena benchmarks tabular causal discovery across synthetic, semantic, formula-grounded, and real-data tracks under shared observational and mixed-interventional protocols. This dashboard separates accuracy, structural error, runtime, scaling, and graph-level inspection so method behavior stays diagnosable beyond one pooled score.

1,200 SCMsFull benchmark: 1000 synthetic, 100 semantic, 100 formula. Public release includes half of each family.

18Methods across control, search, functional, optimization, and pretrained families

600 releasedInitial public release: 500 synthetic, 50 semantic, and 50 formula SCMs; reserved SCMs support held-out evaluation and staged future releases.

13 / 7Real-data checks: 13 obs-only views and 7 interventional datasets

What the arena is designed to test

CausalArena is built around four requirements for causal discovery evaluation in the foundation-model era: broad causal-environment coverage, refreshable benchmark content, grounded causal meaning, and diagnosable failure profiles beyond a single pooled score.

Breadthmany causal settings and difficulty axes

Freshnesspublic plus reserved, evolvable SCM pools

Groundingnamed processes, equations, and interventions

Diagnosabilityslice-level failure profiles, not one score

How to read the dashboard

Use the shared navigation panel for the main leaderboard, semantic/formula case analysis, SCM library, real-data checks, scaling curves, runtime summaries, method metadata, and submission instructions.

Leaderboardmain metrics and cost frontier

Sem/Formulacase-level graph inspection

SCM Libraryreleased grounded SCMs

Supporting viewsreal data, scaling, runtime, methods

Benchmark composition

Track	Scope	Ground truth	What it reveals
Synthetic SCMs	Controlled variation over graph families, mechanisms, root/noise distributions, dimensions, sample sizes, and intervention designs.	Executable DAG + mechanisms	How methods respond to isolated structural and statistical shifts.
Semantic operational SCMs	10 operational domains: cybersecurity and IT operations, education, finance and credit, government services, healthcare delivery, housing and real estate, manufacturing, public health, urban transportation, and water and sanitation.	Executable DAG + variable/edge semantics	Whether methods recover meaningful operational chains, proxies, bottlenecks, and measurement channels.
Formula-grounded SCMs	10 scientific domains: biology and ecology, astronomy, chemistry, mechanics and fluids, earth systems, electromagnetism, energy systems, materials and structures, optics and waves, and thermodynamics.	Executable DAG + equations/ranges	Whether methods handle explicit scientific mechanisms, units, residuals, validity ranges, and equation-level interactions.
Real-data check	Public scored tabular sources: CD-CSG, Sachs flow cytometry, PetShop telemetry, and Causal Chamber physical systems.	Known directed graphs from sources	Whether wrappers still recover known graphs on public real tables outside the generated SCM arena.

Mechanism and distribution taxonomy

This section summarizes the actual category vocabulary present in the current release artifacts. Synthetic categories are read from the generated factor metadata; Semantic and Formula categories are read from the released grounded SCM records. Bars show proportions within the currently released subset where reliable per-category counts are available.

Synthetic SCMs

Controlled generated SCMs with explicit factor axes.

Graph families

bipartite layeredblock sparse layeredchaincollider richdense local moduleserlayeredscale freesmall worldtree

Dimensions

10203050100

10graph families

5dimensions

16edge mechanism families

15noise families

14root distributions

4root-dependency families

Edge mechanisms

activation additiveaggregation nonlineardiscretization assignmentexp loglinear additivelinear additive interactionmixture regimemultiplicativeneural randompiecewise threshold

Child noise

betacensoredexponentialgaussiangumbelheteroskedasticlaplacemixture outliermultiplicative lognormalquantized

Root distributions

bernoullibeta scaledcategorical ordinalexponentialgammagumbellognormalmixture normalmultinomial discretenormal

Root dependency

correlated normalindependent rootsrandom covariance normaluniform ball

All edge mechanisms (16)

activation additiveaggregation nonlineardiscretization assignmentexp loglinear additivelinear additive interactionmixture regimemultiplicativeneural randompiecewise thresholdpolynomial powerrational ratiorff gpsaturation cliptree basedtrig periodic

All noise families (14)

betacensoredexponentialgaussiangumbelheteroskedasticlaplacemixture outliermultiplicative lognormalquantizedrff heteroskedasticstudent ttarget r2 gaussianuniform

All root distributions (14)

bernoullibeta scaledcategorical ordinalexponentialgammagumbellognormalmixture normalmultinomial discretenormalpoissontruncated normaluniformzipf powerlaw

Semantic operational SCMs

Domain-grounded operational graphs; counts below are from released Semantic graph artifacts.

Domains

Cybersecurity & It OperationsEducationFinance & CreditGovernment Services & Policy ImplementationHealthcare DeliveryHousing & Real EstateManufacturingPublic HealthUrban TransportationWater & Sanitation

Edge mechanism proportions

count or event process15%

stock flow or accumulation8.9%

piecewise or threshold8.8%

custom physical law7.4%

categorical or ordinal rule7.1%

mixture or regime switch6.5%

saturating response5.9%

min max or clamp logic5.4%

multiplicative product4.7%

bounded sigmoid index4.3%

ratio or normalization4.2%

deterministic formula3.9%

... plus 48 more categories

Node noise/distribution proportions

normal/Gaussian20%

lognormal15%

count/Bernoulli14%

mixture/zero-inflated10%

quantization/jitter/censoring9.3%

categorical/discrete8.7%

scenario-specific8.4%

beta/proportion8.2%

gamma/exponential2.3%

deterministic/none2.1%

heavy-tail/other continuous1.0%

measurement/calibration<1%

Frequent edge mechanisms

count or event process300stock flow or accumulation184piecewise or threshold181custom physical law152categorical or ordinal rule146mixture or regime switch134saturating response123min max or clamp logic111multiplicative product98bounded sigmoid index88ratio or normalization86deterministic formula80

Node noise/distributions

normal/Gaussian224lognormal176count/Bernoulli164mixture/zero-inflated118quantization/jitter/censoring107categorical/discrete100scenario-specific96beta/proportion94gamma/exponential26deterministic/none24

Root/intervenable distributions

categorical/discrete70normal/Gaussian57count/Bernoulli53beta/proportion49quantization/jitter/censoring49scenario-specific46mixture/zero-inflated44lognormal41deterministic/none15gamma/exponential12

All edge mechanism categories (60)

accumulation rate modifieractuation limitadditive compositionadditive indexaggregation group keyallocation with normalization and capattenuationbaseline headbeta binomial processbounded sigmoid indexcalibration gain offsetcapacity or constraintcategorical effectcategorical or ordinal rulecategorical to random effectcontrol couplingcontrol regimecount or event processcustom physical lawdeclared structural parentdeterministic formulaenvironmental amplificationenvironmental stressexponential decayexponential or logfirst order lagflow scalinghead requirementinteraction effectinteraction modulationlinear affinematerial fatiguemeasurement error modelmin max or clamp logicmixture or regime switchmultiplicative productnonlinear geometrynonmonotonic responseoffset referenceoperating point inertiapiecewise or thresholdpower lawpower scalingpricing policyprobabilistic logitrate reductionratio or normalizationresource constrained delaysampling latencysaturating responseselection censoring processseverity weight to mixturespatial spilloverstiffness increasestock flow or accumulationthrottling lossestriggered policy feedbacktrigonometric or periodicvulnerability amplificationwave propagation

Formula-grounded SCMs

Scientific equation-grounded graphs; counts below are from released Formula graph artifacts.

Scientific domains

Biology Physiology & EcologyCelestial Mechanics & AstronomyChemistry Kinetics & SolutionsClassical Mechanics & Fluid MechanicsEarth Science Hydrology & AtmosphereElectromagnetism & CircuitsEnergy Systems & Environmental EngineeringMaterials & Structural EngineeringOptics Waves & AcousticsThermodynamics & Heat Transfer

Edge mechanism proportions

deterministic formula20%

measurement error model11%

linear affine10%

custom physical law7.6%

calibration gain offset7.0%

ratio or normalization6.2%

piecewise or threshold5.1%

saturating response3.0%

categorical or ordinal rule2.9%

multiplicative product2.8%

min max or clamp logic2.4%

power law2.3%

... plus 75 more categories

Node noise/distribution proportions

normal/Gaussian25%

deterministic/none24%

scenario-specific12%

lognormal8.2%

quantization/jitter/censoring7.1%

categorical/discrete6.6%

heavy-tail/other continuous6.1%

beta/proportion5.1%

mixture/zero-inflated4.2%

count/Bernoulli1.6%

measurement/calibration<1%

gamma/exponential<1%

Frequent edge mechanisms

deterministic formula350measurement error model190linear affine174custom physical law130calibration gain offset120ratio or normalization106piecewise or threshold88saturating response51categorical or ordinal rule49multiplicative product48min max or clamp logic41power law40

Node noise/distributions

normal/Gaussian286deterministic/none272scenario-specific138lognormal95quantization/jitter/censoring82categorical/discrete76heavy-tail/other continuous70beta/proportion59mixture/zero-inflated48count/Bernoulli18

Root/intervenable distributions

normal/Gaussian81categorical/discrete58scenario-specific38lognormal37heavy-tail/other continuous30quantization/jitter/censoring29beta/proportion24deterministic/none23mixture/zero-inflated4gamma/exponential2

Formula families

oxygen content CaO2 physiology1Fick First Law with Stokes Einstein and Beer Lambert1Hardy–Weinberg equilibrium (HWE) allele-to-genotype mapping1logistic growth1Monod growth kinetics with optical response and inverse calibration1Kepler third law with astrometric conversion and barycentric sum1Keplerian circular orbit relations with inverse-μ estimation1Newtonian gravity: escape velocity and surface gravity1Kepler’s third law (two-body) and mean orbital speed1photometric flux pipeline with stellar physics1

All edge mechanism categories (87)

activity correctionadditive indexaging driftalignment logicattenuation multiplicativeaveraging transformbaro alt conversionbeer lambertbeer lambert inversebounded power mapbounded sigmoid indexbuffer equilibriumcalibrated inverse mappingcalibration gaincalibration gain offsetcalibration offsetcapacity modelcategorical offsetcategorical or ordinal rulecontrol plant effectcount or event processcustom physical lawdecision ruledeterministic formuladeterministic physical lawdiagnostic score and flagsdifferencediurnal regime effectdynamic pressureeffective equilibriumelectrochemical nernstelectrode responseenvironmental context modelexponential or logfick first lawgeophysical correctiongradient estimationideal gasimpact stopping distance modelinstrument driftinstrument offsetionic strengthjunction potentialkinematic ratiolinear affinemagnitude transformmatrix effectmeasurement error modelmeasurement quantizationmeasurement saturationmin max or clamp logicmixing dilutionmixture empiricalmixture or regime switchmultiplicative productnonmonotonic responsephoton count processphysical boundpiecewise min controlpiecewise or thresholdpolicy couplingpower lawquadratic drag balancequantized triggerratioratio or normalizationregime mappingsaturating responseschedule arithmeticschedule constraintselection and mappingsigmoid mapsigmoid thresholdstate updatestock flowstock flow or accumulationstoichiometrystokes einsteintemperature effectthermodynamic propertythermodynamic relationthermodynamic transformtrigonometric or periodicunit conversionvalidity constraintweighted fusionwind advection projection

All formula families (50)

Aerodynamic power law (Betz-type, with explicit W→MW scaling)Arrhenius equation (local kinetic response)Beer–Lambert UV–Vis spectrophotometryCombined building heat-loss (transmission + ventilation sensible)Coulomb’s law (electrostatics)Darcy's law for saturated laminar porous flowEuler elastic buckling with rectangular-section geometry and slenderness diagnosticsFick First Law with Stokes Einstein and Beer LambertFourier 1D conduction parallel mergeHardy–Weinberg equilibrium (HWE) allele-to-genotype mappingHenderson–HasselbalchKepler third law with astrometric conversion and barycentric sumKeplerian circular orbit relations with inverse-μ estimationKepler’s third law (two-body) and mean orbital speedLC resonance law with temperature and DC-bias adjusted effective parametersLEFM stress intensity (Mode I) K = Y * sigma * sqrt(pi * a)Monod growth kinetics with optical response and inverse calibrationNOCT thermal model + linear PV temperature deratingNewtonian convection (Newton’s law) and lumped-capacitance transient coolingNewtonian gravity: escape velocity and surface gravityNormal-incidence diffraction grating law with imaging geometry (tan/atan) and inverse wavelength estimationOhm’s law (V = I·R), Joule heating (P = I^2·R), Energy accumulation (E = P·t)RC step response and time constantRL step response and time constantRational Method Q = C·i·A with Tc-coupled IDF and orifice capacity vs. storage attenuationSingle-slit diffraction minima (Fraunhofer)Snell’s law of refraction with Fresnel interface transmission and Beer–Lambert bulk attenuationStefan–Boltzmann longwave radiation and gray-body solar absorptionacoustic doppler effectbeam bending Euler Bernoullicentrifugal pump affinity lawseffectiveness NTU parallel brancheselectrochemistry nernst activityfirst order decay with arrhenius and beer lamberthydropower law with rating and weirhypsometric equation virtual temperaturehypsometric inverseideal projectile motion on level ground with inverse control orientationideal gas energy balancelinear elasticity (Hooke’s law) and strain definitionlinear thermal expansion two zonelogistic growthopen channel flow manningoxygen content CaO2 physiologyphotometric flux pipeline with stellar physicsquadratic drag terminal velocityrolling without slipping energy and geometrysimple harmonic oscillatorsimple pendulum small anglethin lens and magnification plus airy disk

A/B leaderboard

The main table reports aggregate F1/SHD/SID/nSHD together with appendix metrics including precision, recall, AUROC, and AP. The full paper snapshot uses all 1,200 generated SCMs; the public and held-out views split the generated SCM arena into the 600 released SCMs and the reserved 600 SCMs for audited submissions.

Public board: open evaluation on 500 synthetic, 50 semantic, and 50 formula SCMs. Use this for reproducible local testing.

Held-out board: reserved SCMs for audited submissions. Submit predictions/checkpoint metadata to the maintainers; accepted entries update both held-out and all-results views.

Observation-only pairwise win profile

Pairwise win-rate summary over observation-only SCMs and real datasets. Each method pair is compared on both F1 and SHD after averaging repeats; higher F1 or lower SHD is a win, and ties count as 0.5.

Overall benchmark results

All paper-snapshot results

Slice-level behavior

The same method can move substantially across benchmark family and data regime. This heatmap uses the same metric family as the aggregate leaderboard, but keeps Synthetic, Semantic, Formula, and Real-data slices separate.

Per-SCM Semantic/Formula analyzer

Select any released Semantic or Formula SCM and inspect how all methods behave on that particular graph. The table reports replicate-averaged metrics for the selected graph; ΔF1 is method-centered, so positive values mean the method performs better on this SCM than on its own average over released Semantic/Formula SCMs in the same regime.

This is an interactive SCM picker, not a fixed example.Choose a benchmark family, regime, metric, and scenario; the graph, ranking, and error-mode panels update together.

Select SCM below

Benchmark Regime Metric Search Selected SCM scenario

Real-data check

Sources: six CD-CSG observation-only datasets, Sachs flow-cytometry, four PetShop telemetry scenarios, and two Causal Chamber physical-system benchmarks. Obs-only additionally includes observation-only conversions of the seven interventional tables.

Aggregate regime Aggregate metric

Inspect one real dataset.Choose a regime, dataset, and metric; the ranking plot updates accordingly.

Dataset-level ranking

Regime Dataset Metric

Sample-size sensitivity

Synthetic SCMs are fixed at d=30 while observation-only sample size varies. All methods are shown by default; toggle buttons can hide or restore individual curves.

Intervention-protocol sensitivity

The synthetic SCM pool is fixed at d=30; only the intervention design changes. F1 SD summarizes protocol stability.

Runtime and cost

For pretrained/amortized wrappers, the table uses fitted per-graph cost after the one-time model load. For other wrappers, it uses end-to-end per-graph runtime on the synthetic main benchmark.

Submit results or contribute SCMs

Template & schema Validator script Open HF PR / Discussion

Use the same template package for both workflows: method-result submissions live under submission_template/submissions/, while new SCM proposals live under submission_template/scm_contributions/.

Evaluate a new method or upload results

1. Access SCMs

Download the public dataset repository: LAMDA-Tabular/CausalArena.
Use the released 500 synthetic, 50 semantic, and 50 formula SCMs for the public board.

2. Prepare result package

manifest.json: method name, method family, version, implementation URL, checkpoint URL or “N/A”, license, contact, supported regimes, hardware, evaluation date, and training-disclosure notes for possible CausalArena/pretraining overlap.
predictions/: one score matrix and one binary adjacency matrix for each SCM / regime / repeat, with node order and runtime.
summary.csv: graph key, benchmark family, regime, repeat, F1, precision, recall, SHD, nSHD, SID when available, AUROC/AP when applicable, runtime, and status.
Run python tools/validate_submission.py submissions/<method> before sending. The same validator also checks SCM drafts with --scm.

3. Submit through HuggingFace

Open a HuggingFace Dataset pull request or Discussion with the result package, or send a download link if the package is large.
CLI users can submit a draft with hf upload LAMDA-Tabular/CausalArena submissions/<method> submissions/<method> --repo-type dataset --create-pr.
For a maintained evaluation run, provide a runnable wrapper, environment file, checkpoint access instructions, and expected command line.
After validation, maintainers update the public / held-out / full-snapshot leaderboard views and archive the accepted package hash.

Contribute new SCMs

SCM contributions can be lightweight or complete. A minimal graph-only proposal may include just graph.json; richer semantic, formula, or executable benchmark entries can add metadata, mechanisms, interventions, and validation logs.

1. Minimal graph

At minimum, provide graph.json with node order and directed edges. This is enough for proposing a new graph structure or benchmark seed.

2. Optional metadata

Add scm_card.json when available: domain, scenario title, variable descriptions, edge rationales, source notes, ranges, intervention handles, and the intended benchmark role or gap filled.

3. Executable benchmark entry

Add generator_spec.json and validation_report.json when the SCM should be executable under the shared obs-only and mixed-interventional protocols.

Review checklist: graph-only proposals are checked for node/edge validity and acyclicity. Executable SCMs are additionally checked for finite samples, declared ranges and clipping rules, parent mechanisms, noise handling, and protocol compatibility. Semantic/formula metadata is encouraged but not required for every contributed graph.

Method metadata

Parameter counts are meaningful for fixed pretrained checkpoints. Classical and per-dataset optimization methods do not have a single benchmark-wide parameter count.