Bayesian Networks

Foundations and model families

  • Mixed graphical models - Models associations among qualitative and quantitative variables; characterizes decomposable cases. Covers graphs with directed and undirected edges, beyond ordinary Bayesian networks. Lauritzen and Wermuth 1989.
  • Dynamic network forecasting - Extends static belief networks with contemporaneous and lagged dependencies, combining network refinement with time-series forecasting. Dagum et al. 1992.
  • Gaussian networks - Develops Bayesian scores for learning networks of normally distributed variables, using prior-network assessments and likelihood equivalence. Geiger and Heckerman 1994.
  • Learning with Bayesian networks - Heckerman’s tutorial on prior knowledge, parameter estimation, structure scores and incomplete data. Original 1996 manuscript, revised in 2022; arXiv upload dates are later. Heckerman 1998.
  • Dynamic Bayesian networks - Murphy’s thesis covers factored temporal states, exact and approximate inference, and learning; includes hierarchical HMMs and Rao-Blackwellized particle filtering. Murphy 2002.
  • Introducing Bayesian networks - Book chapter on DAGs, conditional probability tables, independence and evidence propagation; from the second edition of Bayesian Artificial Intelligence. Korb and Nicholson 2010.
  • Copula Bayesian networks - Combines a Bayesian-network factorization with copulas to model continuous multivariate densities while separating marginal distributions from dependence. Elidan 2010.

Exact and approximate inference

  • Bucket elimination - Unifies belief updating, most-probable explanations and expected-utility computation through variable elimination; combines elimination with conditioning to trade memory for computation. Dechter 1996.
  • Loopy belief propagation - Tests message passing on graphs with cycles. It can produce useful approximations, but the experiments also show oscillation and inaccurate marginals; convergence alone is not an accuracy certificate. Murphy et al. 1999.
  • Factor graphs and sum-product - Expresses factorizations as bipartite graphs and computes marginal functions by message passing; connects belief propagation, forward-backward and Kalman filtering. Kschischang et al. 2001.
  • Sum-product networks - Uses weighted sums and products to represent distributions with tractable marginals under completeness and consistency conditions; a related alternative to general graphical-model inference. Poon and Domingos 2011.

Independence tests and Markov blankets

  • Equivalence of causal models - Characterizes when graphs encode the same conditional independencies. Observationally equivalent structures limit what independence-based discovery can orient. Verma and Pearl 1990.
  • Grow-Shrink - Grows a candidate Markov blanket by adding conditionally dependent variables, then removes redundant ones. Correctness relies on faithfulness and reliable independence tests. Margaritis 2003.
  • IAMB and variants - Discovers a target’s Markov blanket for variable selection and network learning. Soundness assumes a faithful network and reliable conditional-independence tests. Tsamardinos et al. 2003.
    • Fast-IAMB - Uses speculative additions to accelerate Markov-blanket discovery; the target is a conditionally sufficient variable set for prediction, not a ranking of marginal correlations. Yaramakala and Margaritis 2005.
  • PC in high dimensions - Analyzes PC for sparse Gaussian DAGs under faithfulness and regularity assumptions; conditional-independence testing identifies an equivalence class rather than every causal direction. Kalisch and Bühlmann 2007.

Scores, search and classifiers

  • Prior networks and structure scores - Combines prior-network assessments, confidence and data into Bayesian scores; likelihood-equivalent structures receive compatible scores under the stated assumptions. Heckerman et al. 1995.
  • Tree-augmented naive Bayes - Adds a tree of dependencies among predictors to naive Bayes. Learns that tree using class-conditional mutual information and a maximum-weight spanning tree. Friedman et al. 1997.
  • Sparse Candidate - Alternates restricted network search with refinement of small candidate-parent sets, reducing the number of parent combinations considered on large datasets. Friedman et al. 1999.
  • Greedy equivalence search - Searches equivalence classes through edge additions and deletions; asymptotic correctness depends on the generating model and scoring assumptions, not a finite-sample global optimum guarantee. Chickering 2002.
  • Optimal Reinsertion - Removes a variable and its incident edges, then optimizes its reinsertion. The operator’s optimum does not imply a globally optimal learned network. Moore and Wong 2003.
  • Max-min hill climbing - Combines conditional-independence tests for a candidate skeleton with score-based hill climbing to orient edges; a hybrid of constraint- and score-based structure learning. Tsamardinos et al. 2006.
  • Networks with thousands of variables - Uses inexpensive approximate scores to guide candidate parent-set exploration and improve ordering-based structure search. Scanagatta et al. 2015.
  • Greedy-search computational cost - Analyzes the cost of local distribution fitting within greedy search; counting score evaluations alone can misrepresent runtime across data types and network sizes. Scutari et al. 2019.

Continuous optimization of structure

  • NOTEARS - Recasts linear DAG estimation as continuous optimization with a smooth acyclicity constraint. Acyclicity restricts the graph; it does not establish causal identification from arbitrary data. Zheng et al. 2018.
  • DAG-GNN - Uses a graph-neural variational autoencoder and a differentiable acyclicity constraint for nonlinear structure learning; accommodates discrete and vector-valued variables. Yu et al. 2019.
  • DAGMA - Introduces a log-determinant acyclicity characterization over M-matrices and a central-path optimization scheme, evaluated with linear and nonlinear structural equation models. Bello et al. 2022.

Bayesian structure uncertainty

  • DiBS - Uses a continuous latent representation of graphs for differentiable approximate inference over DAG structures and parameters; retains uncertainty across graphs instead of selecting only one. Lorch et al. 2021.
  • DAG-GFlowNet - Learns to construct DAGs by adding edges with a generative flow network, targeting samples proportional to Bayesian graph scores; approximates a posterior over structures. Deleu et al. 2022.

Parameter estimation and incomplete data

  • EM for network parameters - Alternates inference of expected sufficient statistics with parameter updates. Handles incomplete cases under the model; initialization matters because the likelihood objective can have local maxima. Heckerman 1998.
  • Bayesian Structural EM - Alternates inference over missing or hidden variables with structure search using expected scores. Friedman extends parameter-only EM to Bayesian model selection; convergence does not imply finding the globally best graph. Friedman 1998.
  • Qualitative parameter constraints - Combines qualitative domain constraints with incomplete-data likelihood through a penalty formulation and modified EM, restricting parameter estimates using expert knowledge. Liao and Ji 2009.

Software

  • abn - R package for additive Bayesian networks: local generalized linear models accommodate different response distributions; supports structure search and clustered observations. Kratzer et al. 2023.
  • bnlearn - R tools for structure and parameter learning, discrete/Gaussian/conditional-Gaussian networks, prediction and simulation; includes constraint-based, score-based and hybrid searches. docs.
  • deal - R package for discrete and continuous variables restricted to conditionally Gaussian networks; combines conjugate parameter updating with score-based heuristic structure search. Bøtcher and Dethlefsen 2003.
  • pcalg - R implementations of causal structure discovery and effect estimation, including PC, FCI, GIES and IDA; supported assumptions differ across algorithms. code.
  • BNfinder - Python software for static and dynamic network reconstruction, with classification and continuous-variable scoring extensions. The linked documentation describes a legacy Python 2 implementation. docs.
  • BayesNets.jl - Julia library for representing Bayesian networks, performing inference and learning networks. code.
  • pgmpy - Python tools for probabilistic graphical models, including parameter and structure learning, exact and approximate inference, sampling and causal queries. docs.