🧠 Probabilistic Graphical Models (PGMs)
Research on Bayesian networks, Markov networks, restricted Boltzmann machines, and their applications to machine learning and evolutionary computation.
Probabilistic graphical models (PGMs) provide a principled framework for representing and reasoning about complex probability distributions over many variables. They combine the expressive power of probability theory with the structural efficiency of graph theory. My research on PGMs spans their use in machine learning, estimation of distribution algorithms, and neuroscience, with a focus on learning their structure and parameters from data and applying them to real-world problems.
Bayesian Networks
Bayesian Network Structure Learning
Bayesian networks (BNs) are directed acyclic graphical models that compactly represent joint probability distributions using conditional independence relationships. A central problem in working with Bayesian networks is structure learning: given a dataset, find the directed acyclic graph that best explains the data.
My research has contributed to the development of score-based and constraint-based algorithms for Bayesian network structure learning. Particular attention has been paid to the scalability of these algorithms to high-dimensional datasets and to their use within estimation of distribution algorithms.
Bayesian Networks for Classification
Development and evaluation of Bayesian network classifiers, including naïve Bayes, tree-augmented naïve Bayes (TAN), and general Bayesian network classifiers. Research on how the structure of the Bayesian network affects classification performance and on methods for optimizing the structure for classification tasks specifically.
Bayesian Networks in EDAs
Central use of Bayesian networks as the probabilistic model in estimation of distribution algorithms (EDAs). In this context, the Bayesian network is learned from the selected population in each generation of the EDA and then sampled to produce new candidate solutions. The quality of the Bayesian network directly impacts the efficiency of the evolutionary search.
Markov Networks
Markov Random Fields
Markov networks (also called Markov random fields) are undirected graphical models that represent joint probability distributions through potential functions on cliques. Unlike Bayesian networks, they naturally represent symmetric dependencies between variables, making them suitable for problems where directionality of influence is unclear or irrelevant.
Restricted Boltzmann Machines
Structural Restricted Boltzmann Machines
Research on structural restricted Boltzmann machines (RBMs) as energy-based probabilistic models with applications to image denoising and classification. Structural RBMs extend standard RBMs by allowing structured connections between hidden and visible units, enabling them to capture more complex dependency patterns.
Deep Boltzmann Machines & Deep Belief Networks
Investigation of deep Boltzmann machines and deep belief networks as hierarchical probabilistic models. Research on how the stacking of restricted Boltzmann machines enables learning of increasingly abstract representations of input data, with applications to representation learning and generative modeling.
Structure Learning Algorithms
Score-Based Learning
Development and evaluation of score-based algorithms for learning the structure of probabilistic graphical models from data. These algorithms search over the space of possible graph structures to find the one that maximizes a given scoring function, such as the Bayesian information criterion (BIC) or the Bayesian Dirichlet equivalent (BDe) score.
Vine Copula Structure Learning
Research on algorithms for learning the graph structure of regular vine copulas from dependence data. Vine copulas provide a flexible framework for modeling multivariate dependencies using sequences of bivariate copulas organized in a sequence of trees (the vine structure).
Applications
Density Estimation and Anomaly Detection
Use of probabilistic graphical models for density estimation and anomaly detection in high-dimensional data. By learning a model of the normal data distribution, anomalies can be identified as observations with low probability under the model. Applications to industrial monitoring, network security, and medical diagnosis.
Generation of Synthetic Data
Use of probabilistic graphical models to generate synthetic data that preserves the statistical properties of real data. This is useful for data augmentation, privacy-preserving data release, and the generation of training data for machine learning models when real data is scarce or sensitive.
PGMs for Neuroscience
Application of probabilistic graphical models to model and analyze brain connectivity data. Using PGMs to represent functional and structural connectivity in the brain, with applications to brain decoding, neurofeedback, and the study of cognitive processes.
Selected Publications
- Echegoyen C, Mendiburu A, Santana R and Lozano JA (2012). Toward Understanding EDAs Based on Bayesian Networks Through a Quantitative Analysis. IEEE TEVC.
- Echegoyen C, Lozano JA, Santana R and Larrañaga P (2007). Exact Bayesian network learning in estimation of distribution algorithms. CEC 2007.
- Echegoyen C, Santana R, Lozano JA and Larrañaga P (2008). The impact of probabilistic learning algorithms in EDAs based on Bayesian networks. MEDAL Report.
- Echegoyen C, Mendiburu A, Santana R and Lozano JA (2009). Analyzing the probability of the optimum in EDAs based on Bayesian networks. CEC 2009.
- Echegoyen C, Mendiburu A, Santana R and Lozano JA (2009). A quantitative analysis of estimation of distribution algorithms based on Bayesian networks. GECCO 2009.
- Echegoyen C, Mendiburu A, Santana R and Lozano JA (2010). Estimation of Bayesian networks algorithms in a class of complex networks. GECCO 2010.
- Echegoyen C, Mendiburu A, Santana R and Lozano JA (2010). Analyzing the k most probable solutions in EDAs based on Bayesian networks. LION 2010.
- Larrañaga P, Karshenas H, Bielza C and Santana R (2012). A review on probabilistic graphical models in evolutionary computation. JMLR Workshop.
- Larrañaga P, Karshenas H, Bielza C and Santana R (2013). A Review on Evolutionary Algorithms in Bayesian Network Learning and Inference Tasks. International Journal of Approximate Reasoning.
- Santana R and Shakya S (2012). Probabilistic Graphical Models and Markov Networks. Markov Networks in Evolutionary Computation.
- Santana R (2006). Advances in Probabilistic Graphical Models for Optimization and Learning. Applications in Protein Modelling and EDA Design. PhD Thesis, University of the Basque Country.
- Santana R (2012). MN-EDA and the Use of Clique-Based Factorisations in EDAs. Markov Networks in Evolutionary Computation.
- Zangari-de-Souza M, Santana R, Mendiburu A, Bengoetxea E and Pozo A (2015). MOEA/D-GM: Using probabilistic graphical models in MOEA/D for solving combinatorial optimization problems. CEC 2015.
- Mendiburu A, Santana R and Lozano JA (2007). A parallel framework for loopy belief propagation. GECCO 2007.
- Santana R, Larrañaga P and Lozano JA (2010). Synergies between network-based representations and probabilistic graphical modeling in the solution of combinatorial optimization problems. J. Statistical Mechanics.