| Alba-Cabrera E and Santana R (2010), "Generación de matrices para evaluar el desempeño de estrategias de búsqueda de testores típicos", Avances en Ciencias e Ingenierías. Vol. 2(2), pp. A30-A35. |
| Abstract: Los testores, y en particular los testores típicos, han sido utilizados en problemas de selección de variable y problemas de clasificación supervisada. Comunmente se ha usado algoritmos determinísticos para hallar testores típicos. A principios de esta decada comenzó a desarrollarse un nuevo enfoque basado en algoritmos evolutivos. Un problema común para probar el comportamiento de ambos métodos es la necesidad de conocer a priori el número de testores típicos de una matriz dada. Para una matriz arbitraria, no se puede saber este número a menos de que se hayan encontrado todos los testores típicos. Por lo tanto, este trabajo introduce, por primera vez, una estrategia para generar matrices básicas para las cuales el número de testores típicos es conocido sin necesidad de aplicar un algoritmo para encontrarlos. Este método se ilustra con algunos ejemplos. |
BibTeX:
@article{Alba_and_Santana:2010,
author = {Eduardo Alba-Cabrera and R. Santana},
title = {Generación de matrices para evaluar el desempeño de estrategias de búsqueda de testores típicos},
journal = {Avances en Ciencias e Ingenierías},
year = {2010},
volume = {2},
number = {2},
pages = {A30-A35},
url = {https://revistas.usfq.edu.ec/index.php/avances/article/view/23}
}
|
| Alba-Cabrera E, Santana R, Ochoa A and Lazo-Cortés M (2000), "Finding typical testors by using an evolutionary strategy", In Proceedings of the Fith Ibero American Symposium on Pattern Recognition. Lisbon, Portugal , pp. 267-278. |
| Abstract: The concept of testor appeared in the middle of the fifties. Testors and particularly typical testors, have been used in feature selection and supervised classification problems. Deterministic algorithms have usually been used to find typical testors. In this paper a new approach to find typical testors of a basic matrix is described. This approach is based on the application of the Univariate Marginal Distribution Algorithm as the kernel of a search strategy. The behavior of this algorithm is at least as well as the simple Genetic Algorithms with uniform crossover for the same kind of problems, but it is simpler and less costly in computational terms. Several experiments confirm the validity of this approach. |
BibTeX:
@inproceedings{Alba_et_al:2000,
author = {Eduardo Alba-Cabrera and R. Santana and Alberto Ochoa and Manuel Lazo-Cortés},
title = {Finding typical testors by using an evolutionary strategy},
booktitle = {Proceedings of the Fith Ibero American Symposium on Pattern Recognition},
year = {2000},
pages = {267-278},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/2000/testores}
}
|
| Alvarez-Ginarte YM, Crespo R, Montero-Cabrera LA, Ruiz-Garcia JA, Ponce YM, Santana R, Pardillo-Fontdevila E and Alonso-Becerra E (2005), "A novel in-silico approach for QSAR Studies of Anabolic and Androgenic Activities in the 17-hydroxy-5-androstane Steroid Family", QSAR & Combinatorial Science. Vol. 24, pp. 218-226. |
| Abstract: Predictive Quantitative Structure-Activity Relationship (QSAR) models of anabolic and androgenic activities for the 17β-hydroxy-5α-androstane steroid family were obtained by means of multi-linear regression using quantum and physicochemical molecular descriptors and a genetic algorithm for the selection of the best set of descriptors. The model allows the identification, selection and future design of new steroid molecules with increased anabolic activity. Molecular descriptors included in reported models allow the structural interpretation of the biological process, evidencing the main role of the shape of molecules, hydrophobicity and electronic properties. The model for the anabolic/androgenic ratio (expressed by the weight of the levator ani muscle and ventral prostate in mice) predicts that: a) 2-cyano-17-α-methyl-17-β-acetoxy-5α-androst-2-ene is the most potent anabolic steroid in the group and b) the testosterone-3-cyclopentenyl-enoleter is the less potent one. The approach described in this paper is an alternative for the discovery and optimization of leading anabolic compounds. |
BibTeX:
@article{Alvarez-Ginarte_et_al:2005,
author = {Yoanna María Alvarez-Ginarte and Rachel Crespo and Luis Alberto Montero-Cabrera and José Alberto Ruiz-Garcia and Yovani Marrero Ponce and Roberto Santana and Eladio Pardillo-Fontdevila and Esther Alonso-Becerra},
title = {A novel in-silico approach for QSAR Studies of Anabolic and Androgenic Activities in the 17-hydroxy-5-androstane Steroid Family},
journal = {QSAR & Combinatorial Science},
year = {2005},
volume = {24},
pages = {218--226},
url = {http://dx.doi.org/10.1002/qsar.200430889}
}
|
| Armañanzas R, Inza I, Santana R, Saeys Y, Flores JL, Lozano JA, Van de Peer Y, Blanco R, Robles V, Bielza C and Larrañaga P (2008), "A review of estimation of distribution algorithms in bioinformatics", BioData Mining. Vol. 1(6), pp. doi:10.1186/1756-0381-1-6. |
| Abstract: Evolutionary search algorithms have become an essential asset in the algorithmic toolbox for solving high-dimensional optimization problems in across a broad range of bioinformatics problems. Genetic algorithms, the most well-known and representative evolutionary search technique, have been the subject of the major part of such applications. Estimation of distribution algorithms (EDAs) offer a novel evolutionary paradigm that constitutes a natural and attractive alternative to genetic algorithms. They make use of a probabilistic model, learnt from the promising solutions, to guide the search process. In this paper, we set out a basic taxonomy of EDA techniques, underlining the nature and complexity of the probabilistic model of each EDA variant. We review a set of innovative works that make use of EDA techniques to solve challenging bioinformatics problems, emphasizing the EDA paradigm's potential for further research in this domain. |
BibTeX:
@article{Armananzas_et_al:2008,
author = {R. Armañanzas and I. Inza and R. Santana and Y. Saeys and J. L. Flores and J. A. Lozano and Y. Van de Peer and R. Blanco and V. Robles and C. Bielza and P. Larrañaga},
title = {A review of estimation of distribution algorithms in bioinformatics},
journal = {BioData Mining},
year = {2008},
volume = {1},
number = {6},
pages = {doi:10.1186/1756-0381-1-6},
url = {http://www.biodatamining.org/content/1/1/6}
}
|
| Astigarraga A, Arruti A, Muguerza J, Santana R, Martin JI and Sierra B (2014), "User Adapted Motor-Imaginary Brain-Computer Interface by means of EEG Channel Selection based on Estimation of Distributed Algorithms", Mathematical Problems in Engineering. (151329) |
| Abstract: Brain-Computer Interfaces (BCIs) have become a research field with interesting applications, and it can be inferred from published papers that different persons activate different parts of the brain to perform the same action. This paper presents a personalized interface design method, for electroencephalogram- (EEG-) based BCIs, based on channel selection. We describe a novel two-step method in which firstly a computationally inexpensive greedy algorithm finds an adequate search range; and, then, an Estimation of Distribution Algorithm (EDA) is applied in the reduced range to obtain the optimal channel subset. The use of the EDA allows us to select the most interacting channels subset, removing the irrelevant and noisy ones, thus selecting the most discriminative subset of channels for each user improving accuracy. The method is tested on the IIIa dataset from the BCI competition III. Experimental results show that the resulting channel subset is consistent with motor-imaginary-related neurophysiological principles and, on the other hand, optimizes performance reducing the number of channels. |
BibTeX:
@article{Astigarraga_et_al:2014,
author = {Astigarraga, Aitzol and Arruti, Andoni and Muguerza, Javier and Santana, Roberto and Martin, Jose I and Sierra, Basilio},
title = {User Adapted Motor-Imaginary Brain-Computer Interface by means of EEG Channel Selection based on Estimation of Distributed Algorithms},
journal = {Mathematical Problems in Engineering},
year = {2014},
number = {151329},
url = {https://www.hindawi.com/journals/mpe/2016/1435321/}
}
|
| Carrera D, Santana R and Lozano JA (2016), "Vine copula classifiers for the mind reading problem", Progress in Artificial Intelligence. , pp. 1-17. Springer. |
| Abstract: In this paper we introduce vine copulas to model probabilistic dependencies in supervised classification problems. Vine copulas allow the representation of the dependence structure of multidimensional distributions as a factorization of bivariate pair-copulas. The flexibility of this model lies in the fact that we can mix different types of pair-copulas in a factorization, which allows covering a wide range of types of dependencies, i.e., from independence to much more complex forms of bivariate correlations. This property motivates us to use vine copulas as classifiers, particularly for problems for which the type and strength of bivariate interactions between the variables show a great variability. This is the case of brain signal classification problems where information is represented as multiple time series, each one recorded from different brain region. Our experimental results on a real-word Mind Reading Problem reveal that vine copula-based classifiers perform competitively compared to the four best classification methods presented at the Mind Reading Challenge Competition 2011. |
BibTeX:
@article{Carrera_et_al:2016,
author = {Carrera, Diana and Santana, Roberto and Lozano, Jose A},
title = {Vine copula classifiers for the mind reading problem},
journal = {Progress in Artificial Intelligence},
publisher = {Springer},
year = {2016},
pages = {1--17},
url = {https://link.springer.com/article/10.1007/s13748-016-0095-z}
}
|
| Carrera D, Santana R and Lozano JA (2018), "The Relationship Between Graphical Representations of Regular Vine Copulas and Polytrees", In Proceedings of the 17th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU-2018). Cadiz, Spain , pp. 678-690. |
| Abstract: Graphical models (GMs) are powerful statistical tools for modeling the (in)dependencies among random variables. In this paper, we focus on two different types of graphical models: R-vines and polytrees. Regarding the graphical representation of these models, the former uses a sequence of undirected trees with edges representing pairwise dependencies, whereas the latter uses a directed graph without cycles to encode independence relationships among the variables. The research problem we deal with is whether it is possible to build an R-vine that represents the largest number of independencies found in a polytree and vice versa. Two algorithms are proposed to solve this problem. One algorithm is used to induce an R-vine that represents in each tree the largest number of graphical independencies existing in a polytree. The other one builds a polytree that represents all the independencies found in the R-vine. Through simple examples, both procedures are illustrated. |
BibTeX:
@inproceedings{Carrera_et_al:2018,
author = {D. Carrera and R. Santana and J. A. Lozano},
title = {The Relationship Between Graphical Representations of Regular Vine Copulas and Polytrees},
booktitle = {Proceedings of the 17th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU-2018)},
year = {2018},
pages = {678--690},
url = {https://link.springer.com/chapter/10.1007/978-3-319-91479-4_56}
}
|
| Carrera D, Santana R and Lozano JA (2019), "Detection of sand dunes on Mars using a regular vine-based classification approach", Knowledge Based Systems. Vol. 163, pp. 858-874. |
| Abstract: This paper deals with the problem of detecting sand dunes from remotely sensed images of the surface of Mars. We build on previous approaches that propose methods to extract informative features for the classification of the images. The intricate correlation structure exhibited by these features motivates us to propose the use of probabilistic classifiers based on R-vine distributions to address this problem. R-vines are probabilistic graphical models that combine a set of nested trees with copula functions and are able to model a wide range of pairwise dependencies. We investigate different strategies for building R-vine classifiers and compare them with several state-of-the-art classification algorithms for the identification of Martian dunes. Experimental results show the adequacy of the R-vine-based approach to solve classification problems where the interactions between the variables are of a different nature between classes and play an important role in that the classifier can distinguish the different classes. |
BibTeX:
@article{Carrera_et_al:2019,
author = {Diana Carrera and Roberto Santana and Jose Antonio Lozano},
title = {Detection of sand dunes on Mars using a regular vine-based classification approach},
journal = {Knowledge Based Systems},
year = {2019},
volume = {163},
pages = {858--874},
url = {https://www.sciencedirect.com/science/article/pii/S0950705118304970}
}
|
| Ceberio J, Santana R, Mendiburu A and Lozano JA (2015), "Mixtures of Generalized Mallows models for solving the quadratic assignment problem", In Proceedings of the IEEE Congress on Evolutionary Computation CEC 2015. Sendai, Japan , pp. 2050-2057. IEEE press. |
| Abstract: Recently, distance-based exponential probability models have demonstrated their validity in the context of estimation of distribution algorithms when solving permutation-based combinatorial optimisation problems. However, despite their successful performance, some of these models are unimodal, and, therefore, they might not be flexible enough to model the different modalities that may be represented in heterogeneous populations. In this paper, we address the particular case of the Generalized Mallows models under the Cayley distance, and propose mixtures of these models in the context of estimation of distribution algorithms. In order to evaluate their competitiveness, we considered the quadratic assignment problem as a case of study, and conducted experiments over a set of 90 instances for four different configurations of mixtures. Results reveal that the EDA with mixtures is able to outperform the Generalized Mallows EDA, especially in large instances. |
BibTeX:
@inproceedings{Ceberio_et_al:2015,
author = {J. Ceberio and R. Santana and A. Mendiburu and J. A. Lozano},
title = {Mixtures of Generalized Mallows models for solving the quadratic assignment problem},
booktitle = {Proceedings of the IEEE Congress on Evolutionary Computation CEC 2015},
publisher = {IEEE press},
year = {2015},
pages = {2050-2057},
url = {https://ieeexplore.ieee.org/document/7257137}
}
|
| Cheriet A and Santana R (2018), "Modeling dependencies between decision variables and objectives with copula models", In Proceedings of the Genetic and Evolutionary Computation Conference Companion. , pp. 175-176. |
| Abstract: Probabilistic modeling in multi-objective optimization problems (MOPs) has mainly focused on capturing and representing the dependencies between decision variables in a set of selected solutions. Recently, some works have proposed to model also the dependencies between the objective variables, which are represented as random variables, and the decision variables. In this paper, we investigate the suitability of copula models to capture and exploit these dependencies in MOPs with a continuous representation. Copulas are very flexible probabilistic models able to represent a large variety of probability distributions. |
BibTeX:
@inproceedings{Cheriet_and_Santana:2018,
author = {Cheriet, Abdelhakim and Santana, Roberto},
title = {Modeling dependencies between decision variables and objectives with copula models},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference Companion},
year = {2018},
pages = {175--176},
url = {https://dl.acm.org/doi/10.1145/3205651.3205694}
}
|
| Cosson R, Santana R, Derbel B and Liefooghe A (2022), "Multi-objective NK landscapes with heterogeneous objectives", In Proceedings of the Genetic and Evolutionary Computation Conference. , pp. 502-510. |
| Abstract: So far, multi-objective NK landscapes have been investigated under the assumption of a homogeneous nature of the involved objectives in terms of difficulty. However, we argue that problems with heterogeneous objectives, e.g., in terms of multi-modality, can be challenging for multi-objective evolutionary algorithms, and deserve further considerations. In this paper, we propose a model of multi-objective NK landscapes, where each objective has a different degree of variable interactions (𝐾), as a benchmark to investigate heterogeneous multi-objective optimization problems. We show that the use of a rank-annotated neighborhood network with labeled local optimal solutions, together with landscape metrics extracted from the heterogeneous objectives, thoroughly characterize bi-objective NK landscapes with a different level of heterogeneity among the objectives |
BibTeX:
@inproceedings{Cosson_et_al:2022,
author = {Cosson, Raphaël and Santana, Roberto and Derbel, Bilel and Liefooghe, Arnaud},
title = {Multi-objective NK landscapes with heterogeneous objectives},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference},
year = {2022},
pages = {502--510},
url = {https://dl.acm.org/doi/abs/10.1145/3512290.3528858}
}
|
| Cuesta-Infante A, Santana R, Hidalgo JI, Bielza C and Larrañaga P (2010), "Bivariate empirical and n-variate Archimedean copulas in estimation of distribution algorithms", In Proceedings of the 2010 Congress on Evolutionary Computation CEC-2010. Barcelone, Spain , pp. 1-8. IEEE. |
| Abstract: This paper investigates the use of empirical and Archimedean copulas as probabilistic models of continuous estimation of distribution algorithms (EDAs). A method for learning and sampling empirical bivariate copulas to be used in the context of n-dimensional EDAs is first introduced. Then, by using Archimedean copulas instead of empirical makes possible to construct n-dimensional copulas with the same purpose. Both copula-based EDAs are compared to other known continuous EDAs on a set of 24 functions and different number of variables. Experimental results show that the proposed copula-based EDAs achieve a better behaviour than previous approaches in a 20 percentage of the benchmark functions. |
BibTeX:
@inproceedings{Cuesta_et_al:2010,
author = {Alfredo Cuesta-Infante and Roberto Santana and J. Ignacio Hidalgo and Concha Bielza and Pedro Larrañaga},
title = {Bivariate empirical and n-variate Archimedean copulas in estimation of distribution algorithms},
booktitle = {Proceedings of the 2010 Congress on Evolutionary Computation CEC-2010},
publisher = {IEEE},
year = {2010},
pages = {1-8},
url = {http://dx.doi.org/10.1109/CEC.2010.5586557}
}
|
| Echegoyen C, Lozano JA, Santana R and Larrañaga P (2007), "Exact Bayesian network learning in estimation of distribution algorithms", In Proceedings of the 2007 Congress on Evolutionary Computation CEC-2007. , pp. 1051-1058. IEEE Press. |
| Abstract: This paper introduces exact learning of Bayesian networks in estimation of distribution algorithms. The estimation of Bayesian network algorithm (EBNA) is used to analyze the impact of learning the optimal (exact) structure in the search. By applying recently introduced methods that allow learning optimal Bayesian networks, we investigate two important issues in EDAs. First, we analyze the question of whether learning more accurate (exact) models of the dependencies implies a better performance of EDAs. Second, we are able to study the way in which the problem structure is translated into the probabilistic model when exact learning is accomplished. |
BibTeX:
@inproceedings{Echegoyen_et_al:2007,
author = {C. Echegoyen and J. A. Lozano and R. Santana and P. Larrañaga},
title = {Exact Bayesian network learning in estimation of distribution algorithms},
booktitle = {Proceedings of the 2007 Congress on Evolutionary Computation CEC-2007},
publisher = {IEEE Press},
year = {2007},
pages = {1051--1058},
url = {http://dx.doi.org/10.1109/CEC.2007.4424586}
}
|
| Echegoyen C, Santana R and Lozano JA (2007), "Aprendizaje exacto de redes Bayesianas en algoritmos de estimación de distribuciones", In Actas de las Jornadas de Algoritmos Evolutivos y Metaheurísticas (JAEM I). , pp. 277-284. Thomson.
[BibTeX] |
BibTeX:
@inproceedings{Echegoyen_et_al:2007a,
author = {C. Echegoyen and R. Santana and J. A. Lozano},
editor = {E. Alba and F. Chicano and F. Herrera and F. Luna and G. Luque and A. J. Nebro},
title = {Aprendizaje exacto de redes Bayesianas en algoritmos de estimación de distribuciones},
booktitle = {Actas de las Jornadas de Algoritmos Evolutivos y Metaheurísticas (JAEM I)},
publisher = {Thomson},
year = {2007},
pages = {277--284}
}
|
| Echegoyen C, Santana R, Lozano JA and Larrañaga P (2008), "The impact of probabilistic learning algorithms in EDAs based on Bayesian networks", In Linkage in Evolutionary Computation. , pp. 109-139. Springer. |
| Abstract: This paper discusses exact learning of Bayesian networks in estimation of distribution algorithms. The estimation of Bayesian network algorithm (EBNA) is used to analyze the impact of learning the optimal (exact) structure in the search. By applying recently introduced methods that allow learning optimal Bayesian networks, we investigate two important issues in EDAs. First, we analyze the question of whether learning more accurate (exact) models of the dependencies implies a better performance of EDAs. Secondly, we are able to study the way in which the problem structure is translated into the probabilistic model when exact learning is accomplished. The results obtained reveal that the quality of the problem information captured by the probability model can improve when the accuracy of the learning algorithm employed is increased. However, improvements in model accuracy do not always imply a more efficient search. |
BibTeX:
@inproceedings{Echegoyen_et_al:2008,
author = {C. Echegoyen and R. Santana and J. A. Lozano and P. Larrañaga},
title = {The impact of probabilistic learning algorithms in EDAs based on Bayesian networks},
booktitle = {Linkage in Evolutionary Computation},
publisher = {Springer},
year = {2008},
pages = {109-139},
url = {http://dx.doi.org/10.1007/978-3-540-85068-7_6}
}
|
| Echegoyen C, Santana R, Mendiburu A and Lozano JA (2009), "Estudio de la probabilidad del óptimo en EDAs basados en redes Bayesianas", In Proceedings of the X Congreso Español sobre Metaheurísticas, Algoritmos Evolutivos y Bioinspirados (MAEB-2009). Thomson.
[BibTeX] |
BibTeX:
@inproceedings{Echegoyen_et_al:2008a,
author = {C. Echegoyen and R. Santana and A. Mendiburu and J. A. Lozano},
title = {Estudio de la probabilidad del óptimo en EDAs basados en redes Bayesianas},
booktitle = {Proceedings of the X Congreso Español sobre Metaheurísticas, Algoritmos Evolutivos y Bioinspirados (MAEB-2009)},
publisher = {Thomson},
year = {2009}
}
|
| Echegoyen C, Mendiburu A, Santana R and Lozano JA (2009), "Analyzing the probability of the optimum in EDAs based on Bayesian networks", In Proceedings of the 2009 Congress on Evolutionary Computation CEC-2009. Norway , pp. 1652-1659. IEEE Press. |
| Abstract: In this paper we quantitatively analyze the probability distributions generated by an EDA during the search. In particular, we record the probabilities to the optimal solution, the solution with the highest probability and that of the best individual of the population, when the EDA is solving a trap function. By using different structures in the probabilistic models we can analyze the influence of the structural model accuracy on the aforementioned probability values. In addition, the objective function values of these solutions are contrasted with their probability values in order to study the connection between the function and the probabilistic model. The results provide new information about the behavior of the EDAs and they open a discussion regarding which are the minimum (in)dependences necessary to reach the optimum. |
BibTeX:
@inproceedings{Echegoyen_et_al:2009,
author = {C. Echegoyen and A. Mendiburu and R. Santana and J. A. Lozano},
title = {Analyzing the probability of the optimum in EDAs based on Bayesian networks},
booktitle = {Proceedings of the 2009 Congress on Evolutionary Computation CEC-2009},
publisher = {IEEE Press},
year = {2009},
pages = {1652--1659},
url = {http://dx.doi.org/10.1109/CEC.2009.4983140}
}
|
| Echegoyen C, Mendiburu A, Santana R and Lozano JA (2009), "A quantitative analysis of estimation of distribution algorithms based on Bayesian networks". Research Report at: Department of Computer Science and Artificial Intelligence, University of the Basque Country., October, 2009. (EHU-KZAA-IK-3) |
| Abstract: The successful application of estimation of distribution algorithms (EDAs) to solve different kinds of problems has reinforced their candidature as promising black-box optimization tools. However, their internal behavior is still not completely understood and therefore it is necessary to work in this direction in order to advance their development. This paper presents a new methodology of analysis which provides new information about the behavior of EDAs by quantitatively analyzing the probabilistic models learned during the search. We particularly focus on calculating the probabilities of the optimal solutions, the most probable solution given by the model and the best individual of the population at each step of the algorithm. We carry out the analysis by optimizing functions of different nature such as Trap5, two variants of Ising spin glass and Max-SAT. By using different structures in the probabilistic models, we also analyze the influence of the structural model accuracy in the quantitative behavior of EDAs. In addition, the objective function values of our analyzed key solutions are contrasted with their probability values in order to study the connection between function and probabilistic models. The results not only show information about the EDA behavior, but also about the quality of the optimization process and setup of the parameters, the relationship between the probabilistic model and the fitness function, and even about the problem itself. Furthermore, the results allow us to discover common patterns of behavior in EDAs and propose new ideas in the development of this type of algorithms. |
BibTeX:
@techreport{Echegoyen_et_al:2009a,
author = {C. Echegoyen and A. Mendiburu and R. Santana and J. A. Lozano},
title = {A quantitative analysis of estimation of distribution algorithms based on Bayesian networks},
school = {Department of Computer Science and Artificial Intelligence, University of the Basque Country},
year = {2009},
number = {EHU-KZAA-IK-3},
url = {https://addi.ehu.es/handle/10810/4626}
}
|
| Echegoyen C, Mendiburu A, Santana R and Lozano JA (2010), "Estimation of Bayesian networks algorithms in a class of complex networks", In Proceedings of the 2010 Congress on Evolutionary Computation CEC-2010. Barcelone, Spain IEEE Press. |
| Abstract: In many optimization problems, regardless of the domain to which it belongs, the structural component that the interactions among variables provides can be seen as a network. The impact that the topological characteristics of that network has, both in the hardness of the problem and in the performance of the optimization techniques, constitutes a very important subject of research. In this paper, we study the behavior of estimation of distribution algorithms (EDAs) in functions whose structure is defined by using different network topologies which include grids, small-world networks and random graphs. In order to do that, we use several descriptors such as the population size, the number of evaluations as well as the structures learned during the search. Furthermore, we take measures from the field of complex networks such as clustering coefficient or characteristic path length in order to quantify the topological properties of the function structure and analyze their relation with the behavior of EDAs. The results show that these measures are useful to have better understanding of this type of algorithms which have exhibited a high sensitivity to the topological characteristics of the function structure. This study creates a link between EDAs based on Bayesian networks and the emergent field of complex networks. |
BibTeX:
@inproceedings{Echegoyen_et_al:2010,
author = {C. Echegoyen and A. Mendiburu and R. Santana and J. A. Lozano},
title = {Estimation of Bayesian networks algorithms in a class of complex networks},
booktitle = {Proceedings of the 2010 Congress on Evolutionary Computation CEC-2010},
publisher = {IEEE Press},
year = {2010},
url = {http://dx.doi.org/10.1109/CEC.2010.5586511}
}
|
| Echegoyen C, Mendiburu A, Santana R and Lozano JA (2010), "Analyzing the k most probable solutions in EDAs based on Bayesian networks", In Exploitation of Linkage Learning in Evolutionary Algorithms. , pp. 163-189. Springer. |
| Abstract: Estimation of distribution algorithms (EDAs) have been successfully applied to a wide variety of problems but, for the most complex approaches, there is no clear understanding of the way these algorithms complete the search. For that reason, in this work we exploit the probabilistic models that EDAs based on Bayesian networks are able to learn in order to provide new information about their behavior. Particularly, we analyze the k solutions with the highest probability in the distributions estimated during the search. In order to study the relationship between the probabilistic model and the fitness function, we focus on calculating, for the k most probable solutions (MPSs), the probability values, the function values and the correlation between both sets of values at each step of the algorithm. Furthermore, the objective functions of the k MPSs are contrasted with the k best individuals in the population. We complete the analysis by calculating the position of the optimum in the k MPSs during the search and the genotypic diversity of these solutions. We carry out the analysis by optimizing functions of different natures such as Trap5, two variants of Ising spin glass and Max-SAT. The results not only show information about the relationship between the probabilistic model and the fitness function, but also allow us to observe characteristics of the search space, the quality of the setup of the parameters and even distinguish between successful and unsuccessful runs. |
BibTeX:
@inproceedings{Echegoyen_et_al:2010a,
author = {C. Echegoyen and A. Mendiburu and R. Santana and J. A. Lozano},
title = {Analyzing the k most probable solutions in EDAs based on Bayesian networks},
booktitle = {Exploitation of Linkage Learning in Evolutionary Algorithms},
publisher = {Springer},
year = {2010},
pages = {163-189},
url = {http://dx.doi.org/10.1007/978-3-642-12834-9_8}
}
|
| Echegoyen C, Zhang Q, Mendiburu A, Santana R and Lozano JA (2011), "On the limits of effectiveness in estimation of distribution algorithms", In Proceedings of the 2011 Congress on Evolutionary Computation CEC-2007. , pp. 1573-1580. IEEE Press. |
| Abstract: Which problems a search algorithm can effectively solve is a fundamental issue that plays a key role in understanding and developing algorithms. In order to study the ability limit of estimation of distribution algorithms (EDAs), this paper experimentally tests three different EDA implementations on a sequence of additively decomposable functions (ADFs) with an increasing number of interactions among binary variables. The results show that the ability of EDAs to solve problems could be lost immediately when the degree of variable interaction is larger than a threshold. We argue that this phase-transition phenomenon is closely related with the computational restrictions imposed in the learning step of this type of algorithms. Moreover, we demonstrate how the use of unrestricted Bayesian networks rapidly becomes inefficient as the number of sub-functions in an ADF increases. The study conducted in this paper is useful in order to identify patterns of behavior in EDAs and, thus, improve their performances. |
BibTeX:
@inproceedings{Echegoyen_et_al:2011,
author = {C. Echegoyen and Q. Zhang and A. Mendiburu and R. Santana and J. A. Lozano},
title = {On the limits of effectiveness in estimation of distribution algorithms},
booktitle = {Proceedings of the 2011 Congress on Evolutionary Computation CEC-2007},
publisher = {IEEE Press},
year = {2011},
pages = {1573-1580},
url = {http://dx.doi.org/10.1109/CEC.2011.5949803}
}
|
| Echegoyen C, Zhang Q, Mendiburu A, Santana R and Lozano JA (2011), "Analyzing limits of effectiveness in different implementations of estimation of distribution algorithms networks". Research Report at: Department of Computer Science and Artificial Intelligence, University of the Basque Country. (EHU-KZAA) |
| Abstract: Conducting research in order to know the range of problems in which a search algorithm is effective constitutes a fundamental issue to understand the algorithm and to continue the development of new techniques. In this work, by progressively increasing the degree of interaction in the problem, we study to what extent different EDA implementations are able to reach the optimal solutions. Specifically, we deal with additively decomposable functions whose complexity essentially depends on the number of sub-functions added. With the aim of analyzing the limits of this type of algorithms, we take into account three common EDA implementations that only differ in the complexity of the probabilistic model. The results show that the ability of EDAs to solve problems quickly vanishes after certain degree of interaction with a phase-transition effect. This collapse of performance is closely related with the computational restrictions that this type of algorithms have to impose in the learning step in order to be efficiently applied. Moreover, we show how the use of unrestricted Bayesian networks to solve the problems rapidly becomes inefficient as the number of sub-functions increases. The results suggest that this type of models might not be the most appropriate tool for the the development of new techniques that solve problems with increasing degree of interaction. In general, the experiments proposed in the present work allow us to identify patterns of behavior in EDAs and provide new ideas for the analysis and development of this type of algorithms. |
BibTeX:
@techreport{Echegoyen_et_al:2011a,
author = {C. Echegoyen and Q. Zhang and A. Mendiburu and R. Santana and J. A. Lozano},
title = {Analyzing limits of effectiveness in different implementations of estimation of distribution algorithms networks},
school = {Department of Computer Science and Artificial Intelligence, University of the Basque Country},
year = {2011},
number = {EHU-KZAA},
url = {https://addi.ehu.es/handle/10810/4763}
}
|
| Echegoyen C, Mendiburu A, Santana R and Lozano JA (2012), "Toward Understanding EDAs Based on Bayesian Networks Through a Quantitative Analysis", IEEE Transactions on Evolutionary Computation. Vol. 16(2), pp. 173-189. |
| Abstract: The successful application of estimation of distribution algorithms (EDAs) to solve different kinds of problems has reinforced their candidature as promising black-box optimization tools. However, their internal behavior is still not completely understood and therefore it is necessary to work in this direction in order to advance their development. This paper presents a methodology of analysis which provides new information about the behavior of EDAs by quantitatively analyzing the probabilistic models learned during the search. We particularly focus on calculating the probabilities of the optimal solutions, the most probable solution given by the model and the best individual of the population at each step of the algorithm. We carry out the analysis by optimizing functions of different nature such as Trap5, two variants of Ising spin glass and Max-SAT. By using different structures in the probabilistic models, we also analyze the impact of the structural model accuracy in the quantitative behavior of EDAs. In addition, the objective function values of our analyzed key solutions are contrasted with their probability values in order to study the connection between function and probabilistic models. The results not only show information about the internal behavior of EDAs, but also about the quality of the optimization process and setup of the parameters, the relationship between the probabilistic model and the fitness function, and even about the problem itself. Furthermore, the results allow us to discover common patterns of behavior in EDAs and propose new ideas in the development of this type of algorithms. |
BibTeX:
@article{Echegoyen_et_al:2012a,
author = {C. Echegoyen and A. Mendiburu and R. Santana and J. A. Lozano},
title = {Toward Understanding EDAs Based on Bayesian Networks Through a Quantitative Analysis},
journal = {IEEE Transactions on Evolutionary Computation},
year = {2012},
volume = {16},
number = {2},
pages = {173-189},
url = {http://dx.doi.org/10.1109/TEVC.2010.2102037}
}
|
| Echegoyen C, Mendiburu A, Santana R and Lozano JA (2012), "Clases de equivalencia en algoritmos de estimación de distribuciones", In Proceedings of the VIII Congreso Español sobre Metaheurísticas, Algoritmos Evolutivos y Bioinspirados (MAEB-2012). Albacete |
| Abstract: Entender la relación que surge entre un algoritmo de búsqueda y el espacio de problemas es una cuestión fundamental en el campo de la optimización. En este trabajo nos centramos en la elaboración de taxonomías de problemas para algoritmos de estıtimación de distribuciones (EDAs). Mediante la utilización del modelo de población infinita y asumiendo selección basada en el ranqueo de las soluciones, agrupamos las funciones inyectivas segun el comportamiento del EDA. Para llevar a cabo esta clasificación, se define una relación de equivalencia entre funciones que permite particionar el espacio de funciones en clases de equivalencia para las cuales el algoritmo tiene un comportamiento similar. Considerar diferentes modelos probabilísticos en el EDA genera diferentes particiones del conjunto de posibles problemas. Como consecuencia natural de las definiciones, todas las funciones objetivo están en la misma clase de equivalencia cuando el algoritmo no impone restricciones sobre el modelo probabilístico. Con el fin de crear una primera taxonomía de problemas, nos centramos en la partición que se produce cuando se considera un modelo probabilístico que asume independencia entre las variables. Para ello, primero fijamos las condiciones suficientes para decidir si dos funciones son equivalentes y segundo, obtenemos los operadores para describir y contar los miembros de una clase. En general, el presente trabajo sienta las bases para continuar el estudio del comportamiento de los EDAs y su relación con los problemas de optimización. |
BibTeX:
@inproceedings{Echegoyen_et_al:2012b,
author = {C. Echegoyen and A. Mendiburu and R. Santana and J. A. Lozano},
title = {Clases de equivalencia en algoritmos de estimación de distribuciones},
booktitle = {Proceedings of the VIII Congreso Español sobre Metaheurísticas, Algoritmos Evolutivos y Bioinspirados (MAEB-2012)},
year = {2012},
url = {http://simd.albacete.org/maeb2012/papers/paper_85.pdf}
}
|
| Echegoyen C, Mendiburu A, Santana R and Lozano JA (2013), "On the Taxonomy of Optimization Problems under Estimation of Distribution Algorithms", Evolutionary Computation. Vol. 21(3), pp. 471-495. |
| Abstract: Understanding the relationship between a search algorithm and the space of problems is a fundamental issue in the optimization field. In this paper, we lay the foundations to elaborate taxonomies of problems under estimation of distribution algorithms (EDAs). By using an infinite population model and assuming that the selection operator is based on the rank of the solutions, we group optimization problems according to the behavior of the EDA. Throughout the definition of an equivalence relation between functions it is possible to partition the space of problems in equivalence classes in which the algorithm has the same behavior. We show that only the probabilistic model is able to generate different partitions of the set of possible problems and hence, it predetermines the number of different behaviors that the algorithm can exhibit. As a natural consequence of our definitions, all the objective functions are in the same equivalence class when the algorithm does not impose restrictions to the probabilistic model. The taxonomy of problems, which is also valid for finite populations, is studied in depth for a simple EDA that considers independence among the variables of the problem. We provide the sufficient and necessary condition to decide the equivalence between functions and then we develop the operators to describe and count the members of a class. In addition, we show the intrinsic relation between univariate EDAs and the neighborhood system induced by the Hamming distance by proving that all the functions in the same class have the same number of local optima and that they are in the same ranking positions. Finally, we carry out numerical simulations in order to analyze the different behaviors that the algorithm can exhibit for the functions defined over the search space 0,13. |
BibTeX:
@article{Echegoyen_et_al:2013,
author = {C. Echegoyen and A. Mendiburu and R. Santana and J. A. Lozano},
title = {On the Taxonomy of Optimization Problems under Estimation of Distribution Algorithms},
journal = {Evolutionary Computation},
year = {2013},
volume = {21},
number = {3},
pages = {471-495},
url = {http://www.mitpressjournals.org/doi/abs/10.1162/EVCO_a_00095}
}
|
| Echegoyen C, Santana R, Mendiburu A and Lozano JA (2015), "Comprehensive characterization of the behaviors of estimation of distribution algorithms", Theoretical Computer Science. Vol. 598, pp. 64-86. Elsevier. |
| Abstract: Estimation of distribution algorithms (EDAs) are a successful example of how to use machine learning techniques for designing robust and efficient heuristic search algorithms. Understanding the relationship between EDAs and the space of optimization problems is a fundamental issue for the successful application of this type of algorithms. A step forward in this matter is to create a taxonomy of optimization problems according to the different behaviors that an EDA can exhibit. This paper substantially extends previous work in the proposal of a taxonomy of problems for univariate EDAs, mainly by generalizing those results to EDAs that are able to deal with multivariate dependences among the variables of the problem. Through the definition of an equivalence relation between functions, it is possible to partition the space of problems into equivalence classes in which the algorithm has the same behavior. We provide a sufficient and necessary condition to determine the equivalence between functions. This condition is based on a set of matrices which provides a novel encoding of the relationship between the function and the probabilistic model used by the algorithm. The description of the equivalent functions belonging to a class is studied in depth for EDAs whose probabilistic model is given by a chordal Markov network. Assuming this class of factorization, we unveil the intrinsic connection between the behaviors of EDAs and neighborhood systems defined over the search space. In addition, we carry out numerical simulations that effectively reveal the different behaviors of EDAs for the injective functions defined over the search space . Finally, we provide a novel approach to extend the analysis of equivalence classes to non-injective functions. |
BibTeX:
@article{Echegoyen_et_al:2015,
author = {Echegoyen, Carlos and Santana, Roberto and Mendiburu, Alexander and Lozano, Jose A},
title = {Comprehensive characterization of the behaviors of estimation of distribution algorithms},
journal = {Theoretical Computer Science},
publisher = {Elsevier},
year = {2015},
volume = {598},
pages = {64--86},
url = {https://www.sciencedirect.com/science/article/pii/S0304397515003229}
}
|
| Fontoura VD, Pozo AT and Santana R (2017), "Automated design of hyper-heuristics components to solve the PSP problem with HP model", In 2017 IEEE Congress on Evolutionary Computation (CEC). , pp. 1848-1855. |
| Abstract: The Protein Structure Prediction (PSP) problem is one of the modern most challenging problems from science. Simplified protein models are usually applied to simulate and study some characteristics of the protein folding process. Hence, many heuristic strategies have been applied in order to find simplified protein structures in which the protein configuration has the minimal energy. However, these strategies have difficulties in finding the optimal solutions to the longer sequences of amino-acids, due to the complexity of the problem and the huge amount of local optima. Hyper heuristics have proved to be useful in this type of context since they try to combine different heuristics strengths into a single framework. However, there is lack of work addressing the automated design of hyper-heuristics components. This paper proposes GEHyPSP, an approach which aims to achieve generation, through grammatical evolution, of selection mechanisms and acceptance criteria for a hyper-heuristic framework applied to PSP problem. We investigate the strengths and weaknesses of our approach on a benchmark of simplified protein models. GEHyPSP was able to reach the best known results for 7 instances from 11 that composed the benchmark set used to evaluate the approach. |
BibTeX:
@inproceedings{Fontoura_et_al:2017,
author = {Fontoura, Vidal D and Pozo, Aurora TR and Santana, Roberto},
title = {Automated design of hyper-heuristics components to solve the PSP problem with HP model},
booktitle = {2017 IEEE Congress on Evolutionary Computation (CEC)},
year = {2017},
pages = {1848--1855},
url = {https://ieeexplore.ieee.org/document/7969526}
}
|
| Fritsche G, Strickler A, Pozo A and Santana R (2015), "Capturing Relationships in Multi-Objective Optimization", In Proceedings of the Brazilian Conference on Intelligent Systems (BRACIS 2015). Natal, Brazil , pp. 222-227. |
| Abstract: When applied to multi-objective problems (MOPs), evolutionary algorithms (EAs) can be noticeably improved by representing and exploiting information about the interactions between the components of the problem (variables and objectives). However, accurate detection of such relationships is a challenging question that involves other related issues such as finding the right metric for measuring the interaction, deciding about the timing for testing the interactions, and deciding on appropriate ways to represent the relationships found. In this paper we investigate the performance of three correlation measures (Kendall, Spearman and Pearson) in the context of multi-objective optimization using the MOEA/D-DRA algorithm. We analyze the accuracy of the measures at different stages of the evolution and for different types of relationships. Moreover, the paper proposes a meaningful way for visualizing and interpreting the captured interactions. |
BibTeX:
@inproceedings{Fritsche_et_al:2015,
author = {G. Fritsche and A. Strickler and A. Pozo and R. Santana},
title = {Capturing Relationships in Multi-Objective Optimization},
booktitle = {Proceedings of the Brazilian Conference on Intelligent Systems (BRACIS 2015)},
year = {2015},
pages = {222-227},
url = {https://ieeexplore.ieee.org/document/7424023}
}
|
| Garcia I and Santana R (2021), "Unified Framework for the Analysis of the Effect of Control Policies on Automatic Voltage Regulators", TechRxiv. Vol. 15022611 |
| Abstract: With the advent of smart grids, voltage fluctuation has increased, especially in active distribution networks with a high penetration of distributed energy resources and a large deployment of electric vehicles. In this context, on-load tap-changer (OLTC) distribution transformers have become a key component, mainly because they provide automatic voltage regulation capability. In order to maximise the lifetime of OLTC devices, the number of tap operations should be minimised, avoiding unnecessary changes, but ensuring the main requirement: to keep the voltage within the limits permitted. Therefore, when the automatic mode is active, the control policy followed by the automatic voltage regulator is decisive. This paper presents a novel form of functional approximation of these policies. Furthermore, by means of a unified framework, a methodology for the simulation of policies based on control theory is proposed. The unified framework has been validated using real data. The results confirm the ability of the introduced framework to simulate different scenarios, optimising and validating both existing and new policies by observing their effect on transformer behavior. In addition, it allows the determination of the best-fit policies depending on characteristics such as the pre-selected voltage set point or the voltage variation between transformer taps. |
BibTeX:
@article{Garcia_and_Santana:2021,
author = {Garcia, Iker and Santana, Roberto},
title = {Unified Framework for the Analysis of the Effect of Control Policies on Automatic Voltage Regulators},
journal = {TechRxiv},
year = {2021},
volume = {15022611},
url = {https://www.techrxiv.org/articles/preprint/Unified_Framework_for_the_Analysis_of_the_Effect_of_Control_Policies_on_Automatic_Voltage_Regulators/15022611}
}
|
| Garciarena U and Santana R (2016), "Evolutionary Optimization of Compiler Flag Selection by Learning and Exploiting Flags Interactions", In Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion. New York, NY, USA , pp. 1159-1166. ACM. |
| Abstract: Compiler flag selection can be an effective way to increase the quality of executable code according to different code quality criteria. Evolutionary algorithms have been successfully applied to this optimization problem. However, previous approaches have only partially addressed the question of capturing and exploiting the interactions between compilation options to improve the search. In this paper we deal with this question comparing estimation of distribution algorithms (EDAs) and a traditional genetic algorithm approach. We show that EDAs that learn bivariate interactions can improve the results of GAs for some of the programs considered. We also show that the probabilistic models generated as a result of the search for optimal flag combinations can be used to unveil the (problem-dependent) interactions between the flags, allowing the user a more informed choice of compilation options. |
BibTeX:
@inproceedings{Garciarena_and_Santana:2016,
author = {Garciarena, Unai and Santana, Roberto},
title = {Evolutionary Optimization of Compiler Flag Selection by Learning and Exploiting Flags Interactions},
booktitle = {Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion},
publisher = {ACM},
year = {2016},
pages = {1159--1166},
url = {http://doi.acm.org/10.1145/2908961.2931696},
doi = {10.1145/2908961.2931696}
}
|
| Garciarena U and Santana R (2017), "An extensive analysis of the interaction between missing data types, imputation methods, and supervised classifiers", Expert Systems with Applications. Vol. 89, pp. 52-65. Elsevier. |
| Abstract: When applying data-mining techniques to real-world data, we often find ourselves facing observations that have no value recorded for some attributes. This can be caused by several phenomena, such as a machine’s incapability to record certain characteristics or a person refusing to answer a question in a poll. Depending on that motivation, values gone missing may follow one kind of pattern or another, or describe no regularity at all. One approach to palliate the effect of missing data on machine learning tasks is to replace the missing observations. Imputation algorithms attempt to calculate a value for a missing gap, using information associated with it, i.e., the attribute and/or other values in the same observation. While several imputation methods have been proposed in the literature, few works have addressed the question of the relationship between the type of missing data, the choice of the imputation method, and the effectiveness of classification algorithms that used the imputed data. In this paper we address the relationship among these three factors. By constructing a benchmark of hundreds of databases containing different types of missing data, and applying several imputation methods and classification algorithms, we empirically show that an interaction between imputation methods and supervised classification can be deduced. Besides, differences in terms of classification performance for the same imputation method in different missing data patterns have been found. This points to the convenience of considering the combined choice of the imputation method and the classifier algorithm according to the missing data type. |
BibTeX:
@article{Garciarena_and_Santana:2017,
author = {Garciarena, Unai and Santana, Roberto},
title = {An extensive analysis of the interaction between missing data types, imputation methods, and supervised classifiers},
journal = {Expert Systems with Applications},
publisher = {Elsevier},
year = {2017},
volume = {89},
pages = {52--65},
url = {https://www.sciencedirect.com/science/article/pii/S095741741730502X}
}
|
| Garciarena U, Santana R and Mendiburu A (2017), "Evolving imputation strategies for missing data in classification problems with TPOT", CoRR. Vol. abs/1706.01120 |
| Abstract: Missing data has a ubiquitous presence in real-life applications of machine learning techniques. Imputation methods are algorithms conceived for restoring missing values in the data, based on other entries in the database. The choice of the imputation method has an influence on the performance of the machine learning technique, e.g., it influences the accuracy of the classification algorithm applied to the data. Therefore, selecting and applying the right imputation method is important and usually requires a substantial amount of human intervention. In this paper we propose the use of genetic programming techniques to search for the right combination of imputation and classification algorithms. We build our work on the recently introduced Python-based TPOT library, and incorporate a heterogeneous set of imputation algorithms as part of the machine learning pipeline search. We show that genetic programming can automatically find increasingly better pipelines that include the most effective combinations of imputation methods, feature pre-processing, and classifiers for a variety of classification problems with missing data. |
BibTeX:
@article{Garciarena_et_al:2017a,
author = {Garciarena, Unai and Santana, Roberto and Mendiburu, Alexander},
title = {Evolving imputation strategies for missing data in classification problems with TPOT},
journal = {CoRR},
year = {2017},
volume = {abs/1706.01120},
url = {http://arxiv.org/abs/1706.01120}
}
|
| Garciarena U, Santana R and Mendiburu A (2018), "Evolved GANs for generating Pareto set approximations", In Proceedings of the 2018 on Genetic and Evolutionary Computation Conference. Kyoto, Japan , pp. 434-441. ACM. |
| Abstract: In machine learning, generative models are used to create data samples that mimic the characteristics of the training data. Generative adversarial networks (GANs) are neural-network based generator models that have shown their capacity to produce realistic samples in different domains. In this paper we propose a neuro-evolutionary approach for evolving deep GAN architectures together with the loss function and generator-discriminator synchronization parameters. We also propose the problem of Pareto set (PS) approximation as a suitable benchmark to evaluate the quality of neural-network based generators in terms of the accuracy of the solutions they generate. The covering of the Pareto front (PF) by the generated solutions is used as an indicator of the mode-collapsing behavior of GANs. We show that it is possible to evolve GANs that generate good PS approximations. Our method scales to up to 784 variables and that it is capable to create architecture transferable across dimensions and functions. |
BibTeX:
@inproceedings{Garciarena_et_al:2018,
author = {Unai Garciarena and Roberto Santana and Alexander Mendiburu},
title = {Evolved GANs for generating Pareto set approximations},
booktitle = {Proceedings of the 2018 on Genetic and Evolutionary Computation Conference},
publisher = {ACM},
year = {2018},
pages = {434--441},
url = {https://dl.acm.org/doi/abs/10.1145/3205455.3205550}
}
|
| Garciarena U, Santana R and Mendiburu A (2018), "Expanding variational autoencoders for learning and exploiting latent representations in search distributions", In Proceedings of the 2018 on Genetic and Evolutionary Computation Conference. Kyoto, Japan , pp. 849-856. ACM. |
| Abstract: In the past, evolutionary algorithms (EAs) that use probabilistic modeling of the best solutions incorporated latent or hidden variables to the models as a more accurate way to represent the search distributions. Recently, a number of neural-network models that compute approximations of posterior (latent variable) distributions have been introduced. In this paper, we investigate the use of the variational autoencoder (VAE), a class of neural-network based generative model, for modeling and sampling search distributions as part of an estimation of distribution algorithm. We show that VAE can capture dependencies between decision variables and objectives. This feature is proven to improve the sampling capacity of model based EAs. Furthermore, we extend the original VAE model by adding a new, fitness-approximating network component. We show that it is possible to adapt the architecture of these models and we present evidence of how to extend VAEs to better fulfill the requirements of probabilistic modeling in EAs. While our results are not yet competitive with state of the art probabilistic-based optimizers, they represent a promising direction for the application of generative models within EDAs. |
BibTeX:
@inproceedings{Garciarena_et_al:2018a,
author = {Unai Garciarena and Roberto Santana and Alexander Mendiburu},
title = {Expanding variational autoencoders for learning and exploiting latent representations in search distributions},
booktitle = {Proceedings of the 2018 on Genetic and Evolutionary Computation Conference},
publisher = {ACM},
year = {2018},
pages = {849--856},
url = {https://dl.acm.org/doi/10.1145/3205455.3205645}
}
|
| Garciarena U, Santana R and Mendiburu A (2018), "Analysis of the complexity of the automatic pipeline generation problem", In IEEE Congress on Evolutionary Computation (CEC-2018). Rio de Janeiro, Brazil , pp. 1-8. |
| Abstract: Strategies to automatize the selection of Machine Learning algorithms and their parameters have gained popularity in recent years, to the point of coining the term Automated Machine Learning. The most general version of this problem is pipeline optimization, which seeks an optimal combination of preprocessors and classifiers, along with their respective parameters. In this paper we address the pipeline generation problem from a broader perspective, that of problem complexity understanding as a previous step before proposing a solution, a comprehension we consider critical. The main contribution of this work is the analysis of the characteristics of the fitness landscape. Furthermore, a recently introduced tool for pipeline generation is used to investigate how an automatic method behaves in the previously studied landscape. Results show the high complexity of the pipeline optimization problem, as it can contain several disperse optima, and suffers from a severe lack of generality. Results also suggest that, depending on the dimensions of the search, the model quality target, and the data being modeled, basic search methods can produce results that match the user's expectations. |
BibTeX:
@inproceedings{Garciarena_et_al:2018b,
author = {Unai Garciarena and Roberto Santana and Alexander Mendiburu},
title = {Analysis of the complexity of the automatic pipeline generation problem},
booktitle = {IEEE Congress on Evolutionary Computation (CEC-2018)},
year = {2018},
pages = {1--8},
url = {https://ieeexplore.ieee.org/document/8477662}
}
|
| Garciarena U, Mendiburu A and Santana R (2018), "Towards a more efficient representation of imputation operators in TPOT", CoRR. Vol. abs/1801.04407 |
| Abstract: Automated Machine Learning encompasses a set of meta-algorithms intended to design and apply machine learning techniques (e.g., model selection, hyperparameter tuning, model assessment, etc.). TPOT, a software for optimizing machine learning pipelines based on genetic programming (GP), is a novel example of this kind of applications. Recently we have proposed a way to introduce imputation methods as part of TPOT. While our approach was able to deal with problems with missing data, it can produce a high number of unfeasible pipelines. In this paper we propose a strongly-typed-GP based approach that enforces constraint satisfaction by GP solutions. The enhancement we introduce is based on the redefinition of the operators and implicit enforcement of constraints in the generation of the GP trees. We evaluate the method to introduce imputation methods as part of TPOT. We show that the method can notably increase the efficiency of the GP search for optimal pipelines. |
BibTeX:
@article{Garciarena_et_al:2018c,
author = {Garciarena, Unai and Mendiburu, Alexander and Santana, Roberto},
title = {Towards a more efficient representation of imputation operators in TPOT},
journal = {CoRR},
year = {2018},
volume = {abs/1801.04407},
url = {http://arxiv.org/abs/1801.04407}
}
|
| Garciarena U, Mendiburu A and Santana R (2019), "Towards automatic construction of multi-network models for heterogeneous multi-task learning", CoRR. Vol. abs/1903.09171 |
| Abstract: Multi-task learning, as it is understood nowadays, consists of using one single model to carry out several similar tasks. From classifying hand-written characters of different alphabets to figuring out how to play several Atari games using reinforcement learning, multi-task models have been able to widen their performance range across different tasks, although these tasks are usually of a similar nature. In this work, we attempt to widen this range even further, by including heterogeneous tasks in a single learning procedure. To do so, we firstly formally define a multi-network model, identifying the necessary components and characteristics to allow different adaptations of said model depending on the tasks it is required to fulfill. Secondly, employing the formal definition as a starting point, we develop an illustrative model example consisting of three different tasks (classification, regression and data sampling). The performance of this model implementation is then analyzed, showing its capabilities. Motivated by the results of the analysis, we enumerate a set of open challenges and future research lines over which the full potential of the proposed model definition can be exploited. |
BibTeX:
@article{Garciarena_et_al:2019,
author = {Garciarena, Unai and Mendiburu, Alexander and Santana, Roberto},
title = {Towards automatic construction of multi-network models for heterogeneous multi-task learning},
journal = {CoRR},
year = {2019},
volume = {abs/1903.09171},
url = {http://arxiv.org/abs/1903.09171}
}
|
| Garciarena U, Santana R and Mendiburu A (2018), "EvoFlow: A Python library for evolving deep neural network architectures in tensorflow", In Proceedings of the 2020 IEEE Symposium Series on Computational Intelligence (SSCI-2020). Camberra, Australia , pp. 2288-2295. |
| Abstract: Neuroevolutionary algorithms are one of most effective and extensively applied methods for neural architecture search. While several neuroevolutionary approaches have been proposed, the availability of software that allows a fast development of code to solve problems and test research questions is limited. In this paper we introduce EvoFlow, a Python library for evolving shallow and deep neural network (DNN) architectures. EvoFlow optimizes network structures for DNNs implemented in tensorflow. Single and multi-component DNN architectures are represented by means of descriptors, and the instantiation of the network occurs in the evaluation of the architecture. Genetic operators work by modifying the descriptors. We show how EvoFlow allows efficient architecture optimization of single-component DNNs, such as deep multi-layer perceptrons, but also of multi-component DNNs, such as generative adversarial nets. |
BibTeX:
@inproceedings{Garciarena_et_al:2020,
author = {Unai Garciarena and Roberto Santana and Alexander Mendiburu},
title = {EvoFlow: A Python library for evolving deep neural network architectures in tensorflow},
booktitle = {Proceedings of the 2020 IEEE Symposium Series on Computational Intelligence (SSCI-2020)},
year = {2018},
pages = {2288--2295},
url = {https://ieeexplore.ieee.org/document/9308214}
}
|
| Garciarena U, Mendiburu A and Santana R (2020), "Analysis of the transferability and robustness of GANs evolved for Pareto set approximations", Neural Networks. Vol. 132, pp. 281-296. Elsevier. |
| Abstract: The generative adversarial network (GAN) is a good example of a strong-performing, neural network-based generative model, even though it does have some drawbacks of its own. Mode collapsing and the difficulty in finding the optimal network structure are two of the most concerning issues. In this paper, we address these two issues at the same time by proposing a neuro-evolutionary approach with an agile evaluation method for the fast evolution of robust deep architectures that avoid mode collapsing. The computation of Pareto set approximations with GANs is chosen as a suitable benchmark to evaluate the quality of our approach. Furthermore, we demonstrate the consistency, scalability, and generalization capabilities of the proposed method, which shows its potential applications to many areas. We finally readdress the issue of designing this kind of models by analyzing the characteristics of the best performing GAN specifications, and conclude with a set of general guidelines. This results in a reduction of the many-dimensional problem of structural manual design or automated search. |
BibTeX:
@article{Garciarena_et_al:2020a,
author = {Garciarena, Unai and Mendiburu, Alexander and Santana, Roberto},
title = {Analysis of the transferability and robustness of GANs evolved for Pareto set approximations},
journal = {Neural Networks},
publisher = {Elsevier},
year = {2020},
volume = {132},
pages = {281--296},
url = {https://www.sciencedirect.com/science/article/pii/S0893608020303269}
}
|
| Garciarena U, Mendiburu A and Santana R (2020), "Envisioning the Benefits of Back-Drive in Evolutionary Algorithms", In 2020 IEEE Congress on Evolutionary Computation (CEC). , pp. 1-8. |
| Abstract: Reliable deployment of machine learning models such as neural networks continues to be challenging due to several limitations. Some of the main shortcomings are the lack of interpretability and the lack of robustness against adversarial examples or out-of-distribution inputs. In this paper, we explore the possibilities and limits of adversarial attacks for explainable machine learning models. First, we extend the notion of adversarial examples to fit in explainable machine learning scenarios, in which the inputs, the output classifications and the explanations of the model's decisions are assessed by humans. Next, we propose a comprehensive framework to study whether (and how) adversarial examples can be generated for explainable models under human assessment, introducing novel attack paradigms. In particular, our framework considers a wide range of relevant (yet often ignored) factors such as the type of problem, the user expertise or the objective of the explanations in order to identify the attack strategies that should be adopted in each scenario to successfully deceive the model (and the human). These contributions intend to serve as a basis for a more rigorous and realistic study of adversarial examples in the field of explainable machine learning. |
BibTeX:
@inproceedings{Garciarena_et_al:2020b,
author = {Garciarena, Unai and Mendiburu, Alexander and Santana, Roberto},
title = {Envisioning the Benefits of Back-Drive in Evolutionary Algorithms},
booktitle = {2020 IEEE Congress on Evolutionary Computation (CEC)},
year = {2020},
pages = {1--8},
url = {https://arxiv.org/abs/2107.01943}
}
|
| Garciarena U, Mendiburu A and Santana R (2020), "Automatic Structural Search for Multi-task Learning VALPs", In International Conference on Optimization and Learning. , pp. 25-36. |
| Abstract: The neural network research field is still producing novel and improved models which continuously outperform their predecessors. However, a large portion of the best-performing architectures are still fully hand-engineered by experts. Recently, methods that automatize the search for optimal structures have started to reach the level of state-of-the-art hand-crafted structures. Nevertheless, replacing the expert knowledge requires high efficiency from the search algorithm, and flexibility on the part of the model concept. This work proposes a set of model structure-modifying operators designed specifically for the VALP, a recently introduced multi-network model for heterogeneous multi-task problems. These modifiers are employed in a greedy multi-objective search algorithm which employs a non dominance-based acceptance criterion in order to test the viability of a structure-exploring method built on the operators. The results obtained from the experiments carried out in this work indicate that the modifiers can indeed form part of intelligent searches over the space of VALP structures, which encourages more research in this direction. |
BibTeX:
@inproceedings{Garciarena_et_al:2020c,
author = {Garciarena, Unai and Mendiburu, Alexander and Santana, Roberto},
title = {Automatic Structural Search for Multi-task Learning VALPs},
booktitle = {International Conference on Optimization and Learning},
year = {2020},
pages = {25--36},
url = {https://link.springer.com/chapter/10.1007/978-3-030-41913-4_3}
}
|
| Garciarena U, Mendiburu A and Santana R (2021), "Towards automatic construction of multi-network models for heterogeneous multi-task learning", ACM Transactions on Knowledge Discovery from Data (TKDD). Vol. 15(2), pp. 1-23. ACM New York, NY, USA. |
| Abstract: Multi-task learning, as it is understood nowadays, consists of using one single model to carry out several similar tasks. From classifying hand-written characters of different alphabets to figuring out how to play several Atari games using reinforcement learning, multi-task models have been able to widen their performance range across different tasks, although these tasks are usually of a similar nature. In this work, we attempt to expand this range even further, by including heterogeneous tasks in a single learning procedure. To do so, we firstly formally define a multi-network model, identifying the necessary components and characteristics to allow different adaptations of said model depending on the tasks it is required to fulfill. Secondly, employing the formal definition as a starting point, we develop an illustrative model example consisting of three different tasks (classification, regression, and data sampling). The performance of this illustrative model is then analyzed, showing its capabilities. Motivated by the results of the analysis, we enumerate a set of open challenges and future research lines over which the full potential of the proposed model definition can be exploited. |
BibTeX:
@article{Garciarena_et_al:2021,
author = {Garciarena, Unai and Mendiburu, Alexander and Santana, Roberto},
title = {Towards automatic construction of multi-network models for heterogeneous multi-task learning},
journal = {ACM Transactions on Knowledge Discovery from Data (TKDD)},
publisher = {ACM New York, NY, USA},
year = {2021},
volume = {15},
number = {2},
pages = {1--23},
url = {https://dl.acm.org/doi/abs/10.1145/3434748}
}
|
| Garciarena U, Santana R and Mendiburu A (2021), "Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components", CoRR. Vol. abs/2106.08972 |
| Abstract: With neural architecture search methods gaining ground on manually designed deep neural networks -even more rapidly as model sophistication escalates-, the research trend shifts towards arranging different and often increasingly complex neural architecture search spaces. In this conjuncture, delineating algorithms which can efficiently explore these search spaces can result in a significant improvement over currently used methods, which, in general, randomly select the structural variation operator, hoping for a performance gain. In this paper, we investigate the effect of different variation operators in a complex domain, that of multi-network heterogeneous neural models. These models have an extensive and complex search space of structures as they require multiple sub-networks within the general model in order to answer to different output types. From that investigation, we extract a set of general guidelines, whose application is not limited to that particular type of model, and are useful to determine the direction in which an architecture optimization method could find the largest improvement. To deduce the set of guidelines, we characterize both the variation operators, according to their effect on the complexity and performance of the model; and the models, relying on diverse metrics which estimate the quality of the different parts composing it. |
BibTeX:
@article{Garciarena_et_al:2021b,
author = {Garciarena, Unai and Santana, Roberto and Mendiburu, Alexander},
title = {Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components},
journal = {CoRR},
year = {2021},
volume = {abs/2106.08972},
url = {http://arxiv.org/abs/2106.08972}
}
|
| Garciarena U, Lourenço N, Machado P, Santana R and Mendiburu A (2021), "On the Exploitation of Neuroevolutionary Information: Analyzing the Past for a More Efficient Future", CoRR. Vol. abs/2105.12836 |
| Abstract: Neuroevolutionary algorithms, automatic searches of neural network structures by means of evolutionary techniques, are computationally costly procedures. In spite of this, due to the great performance provided by the architectures which are found, these methods are widely applied. The final outcome of neuroevolutionary processes is the best structure found during the search, and the rest of the procedure is commonly omitted in the literature. However, a good amount of residual information consisting of valuable knowledge that can be extracted is also produced during these searches. In this paper, we propose an approach that extracts this information from neuroevolutionary runs, and use it to build a metamodel that could positively impact future neural architecture searches. More specifically, by inspecting the best structures found during neuroevolutionary searches of generative adversarial networks with varying characteristics (e.g., based on dense or convolutional layers), we propose a Bayesian network-based model which can be used to either find strong neural structures right away, conveniently initialize different structural searches for different problems, or help future optimization of structures of any type to keep finding increasingly better structures where uninformed methods get stuck into local optima. |
BibTeX:
@article{Garciarena_et_al:2021c,
author = {Garciarena, Unai and Lourenço, Nuno and Machado, Penousal and Santana, Roberto and Mendiburu, Alexander},
title = {On the Exploitation of Neuroevolutionary Information: Analyzing the Past for a More Efficient Future},
journal = {CoRR},
year = {2021},
volume = {abs/2105.12836},
url = {http://arxiv.org/abs/2105.12836}
}
|
| Garciarena U, Vadillo J, Mendiburu A and Santana R (2021), "Adversarial Perturbations for Evolutionary Optimization", In International Conference on Machine Learning, Optimization, and Data Science (LOD-2021). Vol. 13164, pp. 408-422. |
| Abstract: Sampling methods are a critical step for model-based evolutionary algorithms, their goal being the generation of new and promising individuals based on the information provided by the model. Adversarial perturbations have been proposed as a way to create samples that deceive neural networks. In this paper we introduce the idea of creating adversarial perturbations that correspond to promising solutions of the search space. A surrogate neural network is “fooled” by an adversarial perturbation algorithm until it produces solutions that are likely to be of higher fitness than the present ones. Using a benchmark of functions with varying levels of difficulty, we investigate the performance of a number of adversarial perturbation techniques as sampling methods. The paper also proposes a technique to enhance the effect that adversarial perturbations produce in the network. While adversarial perturbations on their own are not able to produce evolutionary algorithms that compete with state of the art methods, they provide a novel and promising way to combine local optimizers with evolutionary algorithms. |
BibTeX:
@inproceedings{Garciarena_et_al:2021d,
author = {Garciarena, U. and Vadillo, J. and Mendiburu, A. and Santana, R.},
title = {Adversarial Perturbations for Evolutionary Optimization},
booktitle = {International Conference on Machine Learning, Optimization, and Data Science (LOD-2021)},
year = {2021},
volume = {13164},
pages = {408--422},
url = {https://link.springer.com/chapter/10.1007/978-3-030-95470-3_31}
}
|
| Garcia Rodriguez MJ, Rodriguez Montequin V, Aranguren Ubierna A, Santana R, Sierra Araujo B and Zelaia Jauregi A (2021), "Award Price Estimator for Public Procurement Auctions Using Machine Learning Algorithms: Case Study with Tenders from Spain", Studies in Informatics and Control. Vol. 30(4), pp. 67-76. |
| Abstract: The public procurement process plays an important role in the efficient use of public resources. In this context, the evaluation of machine learning techniques that are able to predict the award price is a relevant research topic. In this paper, the suitability of a representative set of machine learning algorithms is evaluated for this problem. The traditional regression methods, such as linear regression and random forest, are compared with the less investigated paradigms, such as isotonic regression and popular artificial neural network models. Extensive experiments are conducted based on the Spanish public procurement announcements (tenders) dataset and employ diverse error metrics and implementations in WEKA and Tensorflow 2. |
BibTeX:
@article{GarciaRodriguez_et_al:2021,
author = {Garcia Rodriguez, Manuel J and Rodriguez Montequin, Vicente and Aranguren Ubierna, Andoni and Santana, Roberto and Sierra Araujo, Basilio and Zelaia Jauregi, Ana},
title = {Award Price Estimator for Public Procurement Auctions Using Machine Learning Algorithms: Case Study with Tenders from Spain},
journal = {Studies in Informatics and Control},
year = {2021},
volume = {30},
number = {4},
pages = {67--76},
url = {https://www.researchgate.net/publication/357065176_Award_Price_Estimator_for_Public_Procurement_Auctions_Using_Machine_Learning_Algorithms_Case_Study_with_Tenders_from_Spain}
}
|
| González-Arenas Z, Jiménez-Sobrino JC, Lozada-Chang L and Santana R (2008), "Parameter estimation of diffusion processes using EDAs", In Proceedings of VIII International Conference on Operations Research. La Habana, Cuba , pp. 55. |
| Abstract: In this work we introduced a specific EDA, a continuous version of the Univariate Marginal Distribution Algorithm (UMDAc), to seek a solution for the optimization problem related with the estimation of unknown parameters in a diffusion process. There were considered two different examples for evaluating our proposal behavior and a comparison was made with the case of using a local search algorithm. |
BibTeX:
@inproceedings{Gonzalez_et_al::2008,
author = {Z. González-Arenas and J. C. Jiménez-Sobrino and L. Lozada-Chang and R. Santana},
title = {Parameter estimation of diffusion processes using EDAs},
booktitle = {Proceedings of VIII International Conference on Operations Research},
year = {2008},
pages = {55}
}
|
| Arenas ZG, Jimenez JC, Lozada-Chang L-V and Santana R (2021), "Estimation of distribution algorithms for the computation of innovation estimators of diffusion processes", Mathematics and Computers in Simulation. Vol. 187, pp. 449-467. Elsevier. |
| Abstract: Innovation Method is a recognized method for the estimation of parameters in diffusion processes. It is well known that the quality of the Innovation Estimator strongly depends on an adequate selection of the initial value for the parameters when a local optimization algorithm is used in its computation. Alternatively, in this paper, we use a strategy based on a modern method for solving global optimization problems, Estimation of Distribution Algorithms (EDAs). We study the feasibility of a specific EDA - a continuous version of the Univariate Marginal Distribution Algorithm (UMDAc) - for the computation of the Innovation Estimators. Through numerical simulations, we show that the considered global optimization algorithms substantially improves the effectiveness of the Innovation Estimators for different types of diffusion processes with complex nonlinear and stochastic dynamics. |
BibTeX:
@article{GonzalezArenas_et_al:2021,
author = {Arenas, Zochil González and Jimenez, Juan Carlos and Lozada-Chang, Li-Vang and Santana, Roberto},
title = {Estimation of distribution algorithms for the computation of innovation estimators of diffusion processes},
journal = {Mathematics and Computers in Simulation},
publisher = {Elsevier},
year = {2021},
volume = {187},
pages = {449--467},
url = {https://www.sciencedirect.com/science/article/pii/S0378475421000926}
}
|
| Höns R, Santana R, Larrañaga P and Lozano JA (2007), "Optimization by max-propagation using Kikuchi approximations". Research Report at: Department of Computer Science and Artificial Intelligence, University of the Basque Country., November, 2007. (EHU-KZAA-IK-2/07) |
| Abstract: In this paper we address the problem of using region-based approximations to find the optimal points of a given function. Our approach combines the use of Kikuchi approximations with the application of generalized belief propagation (GBP) using maximization instead of marginalization. The relationship between the fixed points of maximum GBP and the free energy is elucidated. A straightforward connection between the function to be optimized and the Kikuchi approximation (which holds only for maximum GBP, not for marginal GBP) is proven. Later, we show that maximum GBP can be combined with a dynamic programming algorithm to find the most probable configurations of a graphical model. We then analyze the dynamics of the procedure proposed and show how its different steps can be manipulated to influence the search for optimal solutions. |
BibTeX:
@techreport{Hoens_et_al:2007,
author = {R. Höns and R. Santana and P. Larrañaga and J. A. Lozano},
title = {Optimization by max-propagation using Kikuchi approximations},
school = {Department of Computer Science and Artificial Intelligence, University of the Basque Country},
year = {2007},
number = {EHU-KZAA-IK-2/07},
url = {https://www.researchgate.net/publication/228923642_Optimization_by_max-propagation_using_Kikuchi_approximations}
}
|
| Irurozki E, Ceberio J, Santamaria J, Santana R and Mendiburu A (2018), "Algorithm 989: perm_mateda: A Matlab Toolbox of Estimation of Distribution Algorithms for Permutation-based Combinatorial Optimization Problems", ACM Transactions on Mathematical Software (TOMS). Vol. 44(4), pp. 47. ACM. |
| Abstract: Permutation problems are combinatorial optimization problems whose solutions are naturally codified as permutations. Due to their complexity, motivated principally by the factorial cardinality of the search space of solutions, they have been a recurrent topic for the artificial intelligence and operations research community. Recently, among the vast number of metaheuristic algorithms, new advances on estimation of distribution algorithms (EDAs) have shown outstanding performance when solving some permutation problems. These novel EDAs implement distance-based exponential probability models such as the Mallows and Generalized Mallows models. In this article, we present a Matlab package, perm_mateda, of estimation of distribution algorithms on permutation problems, which has been implemented as an extension to the Mateda-2.0 toolbox of EDAs. Particularly, we provide implementations of the Mallows and Generalized Mallows EDAs under the Kendall’s-τ, Cayley, and Ulam distances. In addition, four classical permutation problems have also been implemented: Traveling Salesman Problem, Permutation Flowshop Scheduling Problem, Linear Ordering Problem, and Quadratic Assignment Problem. |
BibTeX:
@article{Irurozki_et_al:2018,
author = {Ekhine Irurozki and Josu Ceberio and Josean Santamaria and Roberto Santana and Alexander Mendiburu},
title = {Algorithm 989: perm_mateda: A Matlab Toolbox of Estimation of Distribution Algorithms for Permutation-based Combinatorial Optimization Problems},
journal = {ACM Transactions on Mathematical Software (TOMS)},
publisher = {ACM},
year = {2018},
volume = {44},
number = {4},
pages = {47},
url = {https://dl.acm.org/doi/abs/10.1145/3206429}
}
|
| Karshenas H, Santana R, Bielza C and Larrañaga P (2011), "Multi-objective optimization with joint probabilistic modeling of objectives and variables", In Evolutionary Multi-Criterion Optimization: Sixth International Conference, EMO 2011. , pp. 298-312. Springer Berlin-Heidelberg. |
| Abstract: The objective values information can be incorporated into the evolutionary algorithms based on probabilistic modeling in order to capture the relationships between objectives and variables. This paper investigates the effects of joining the objective and variable information on the performance of an estimation of distribution algorithm for multi-objective optimization. A joint Gaussian Bayesian network of objectives and variables is learnt and then sampled using the information about currently best obtained objective values as evidence. The experimental results obtained on a set of multi-objective functions and in comparison to two other competitive algorithms are presented and discussed. |
BibTeX:
@inproceedings{Karshenas_et_al:2011,
author = {H. Karshenas and R. Santana and C. Bielza and P. Larrañaga},
title = {Multi-objective optimization with joint probabilistic modeling of objectives and variables},
booktitle = {Evolutionary Multi-Criterion Optimization: Sixth International Conference, EMO 2011},
publisher = {Springer Berlin-Heidelberg},
year = {2011},
pages = {298--312},
url = {http://dx.doi.org/10.1007/978-3-642-19893-9_21}
}
|
| Karshenas H, Santana R, Bielza C and Larrañaga P (2011), "Regularized Model Learning in Estimation of Distribution Algorithms for Continuous Optimization Problems". Research Report at: Department of Artificial Intelligence, Faculty of Informatics, Technical University of Madrid., January, 2011. (UPM-FI/DIA/2011-1) |
| Abstract: Regularization is a well-known technique in statistics for model estimation which is used to improve the generalization ability of the estimated model. Some of the regularization methods can also be used for variable selection that is especially useful in high-dimensional problems. This paper studies the use of regularized model learning in estimation of distribution algorithms (EDAs) for continuous optimization based on Gaussian distributions. We introduce two approaches to the regularized model estimation and analyze their effect on the accuracy and computational complexity of model learning in EDAs. We then apply the proposed algorithms to a number of continuous optimization functions and compare their results with other Gaussian distribution-based EDAs. The results show that the optimization performance of the proposed RegEDAs is less affected by the increase in the problem size than other EDAs, and they are able to obtain significantly better optimization values for many of the functions in high-dimensional settings. |
BibTeX:
@techreport{Karshenas_et_al:2011a,
author = {H. Karshenas and R. Santana and C. Bielza and P. Larrañaga},
title = {Regularized Model Learning in Estimation of Distribution Algorithms for Continuous Optimization Problems},
school = {Department of Artificial Intelligence, Faculty of Informatics, Technical University of Madrid},
year = {2011},
number = {UPM-FI/DIA/2011-1},
url = {https://www.sciencedirect.com/science/article/pii/S1568494612005376}
}
|
| Karshenas H, Santana R, Bielza C and Larrañaga P (2012), "Continuous estimation of distribution algorithms based on factorized Gaussian Markov networks", In Markov Networks in Evolutionary Computation. , pp. 157-173. Springer. |
| Abstract: Because of their intrinsic properties, the majority of the estimation of distribution algorithms proposed for continuous optimization problems are based on the Gaussian distribution assumption for the variables. This paper looks over the relation between the general multivariate Gaussian distribution and the popular undirected graphical model of Markov networks and discusses how they can be employed in estimation of distribution algorithms for continuous optimization. A number of learning and sampling techniques for thesemodels, including the promising regularized model learning, are also reviewed and their application for function optimization in the context of estimation of distribution algorithms is studied. |
BibTeX:
@incollection{Karshenas_et_al:2012,
author = {H. Karshenas and R. Santana and C. Bielza and P. Larrañaga},
editor = {S. Shakya and R. Santana},
title = {Continuous estimation of distribution algorithms based on factorized Gaussian Markov networks},
booktitle = {Markov Networks in Evolutionary Computation},
publisher = {Springer},
year = {2012},
pages = {157-173},
url = {http://dx.doi.org/10.1007/978-3-642-28900-2_10}
}
|
| Karshenas H, Santana R, Bielza C and Larrañaga P (2012), "Multi-objective optimization based on joint probabilistic modeling of objectives and variables". Research Report at: Department of Artificial Intelligence, Faculty of Informatics, Technical University of Madrid. (UPM-FI/DIA/2012-2) |
| Abstract: This paper proposes a new multi-objective estimation of distribution algorithm (EDA) based on joint modeling of objectives and variables. This EDA uses the multi-dimensional Bayesian network as its probabilistic model. In this way it can capture the dependencies between objectives, variables and objectives, as well as the dependencies learnt between variables in other Bayesian network-based EDAs. This model leads to a problem decomposition that helps the proposed algorithm to find better trade-off solutions to the multi-objective problem. In addition to Pareto set approximation, the algorithm is also able to estimate the structure of the multi-objective problem. To apply the algorithm to many-objective problems, the algorithm includes four different ranking methods proposed in the literature for this purpose. The algorithm is applied to the set of walking fish group (WFG) problems, and its optimization performance is compared with an evolutionary algorithm and another multi-objective EDA. The experimental results show that the proposed algorithm performs significantly better on many of the problems and for different objective space dimensions, and achieves comparable results on some compared with the other algorithms. |
BibTeX:
@techreport{Karshenas_et_al:2012b,
author = {H. Karshenas and R. Santana and C. Bielza and P. Larrañaga},
title = {Multi-objective optimization based on joint probabilistic modeling of objectives and variables},
school = {Department of Artificial Intelligence, Faculty of Informatics, Technical University of Madrid},
year = {2012},
number = {UPM-FI/DIA/2012-2},
url = {https://link.springer.com/chapter/10.1007/978-3-642-19893-9_21}
}
|
| Karshenas H, Santana R, Bielza C and Larrañaga P (2013), "Regularized Continuous Estimation of Distribution Algorithms", Applied Soft Computing. Vol. 13(5), pp. 2412-2432. |
| Abstract: Regularization is a well-known technique in statistics for model estimation which is used to improve the generalization ability of the estimated model. Some of the regularization methods can also be used for variable selection that is especially useful in high-dimensional problems. This paper studies the use of regularized model learning in estimation of distribution algorithms (EDAs) for continuous optimization based on Gaussian distributions. We introduce two approaches to the regularized model estimation and analyze their effect on the accuracy and computational complexity of model learning in EDAs. We then apply the proposed algorithms to a number of continuous optimization functions and compare their results with other Gaussian distribution-based EDAs. The results show that the optimization performance of the proposed RegEDAs is less affected by the increase in the problem size than other EDAs, and they are able to obtain significantly better optimization values for many of the functions in high-dimensional settings. |
BibTeX:
@article{Karshenas_et_al:2013,
author = {H. Karshenas and R. Santana and C. Bielza and P. Larrañaga},
title = {Regularized Continuous Estimation of Distribution Algorithms},
journal = {Applied Soft Computing},
year = {2013},
volume = {13},
number = {5},
pages = {2412--2432},
url = {https://www.sciencedirect.com/science/article/pii/S1568494612005376}
}
|
| Karshenas H, Santana R, Bielza C and Larrañaga P (2014), "Multiobjective Estimation of Distribution Algorithm Based on Joint Modeling of Objectives and Variables", IEEE Transactions on Evolutionary Computation. Vol. 18(4), pp. 519-542. |
| Abstract: This paper proposes a new multiobjective estimation of distribution algorithm (EDA) based on joint probabilistic modeling of objectives and variables. This EDA uses the multidimensional Bayesian network as its probabilistic model. In this way, it can capture the dependencies between objectives, variables and objectives, as well as the dependencies learned between variables in other Bayesian network-based EDAs. This model leads to a problem decomposition that helps the proposed algorithm find better tradeoff solutions to the multiobjective problem. In addition to Pareto set approximation, the algorithm is also able to estimate the structure of the multiobjective problem. To apply the algorithm to many-objective problems, the algorithm includes four different ranking methods proposed in the literature for this purpose. The algorithm is first applied to the set of walking fish group problems, and its optimization performance is compared with a standard multiobjective evolutionary algorithm and another competitive multiobjective EDA. The experimental results show that on several of these problems, and for different objective space dimensions, the proposed algorithm performs significantly better and on some others achieves comparable results when compared with the other two algorithms. The algorithm is then tested on the set of CEC09 problems, where the results show that multiobjective optimization based on joint model estimation is able to obtain considerably better fronts for some of the problems compared with the search based on conventional genetic operators in the state-of-the-art multiobjective evolutionary algorithms. |
BibTeX:
@article{Karshenas_et_al:2014,
author = {H. Karshenas and R. Santana and C. Bielza and P. Larrañaga},
title = {Multiobjective Estimation of Distribution Algorithm Based on Joint Modeling of Objectives and Variables},
journal = {IEEE Transactions on Evolutionary Computation},
year = {2014},
volume = {18},
number = {4},
pages = {519--542},
url = {https://ieeexplore.ieee.org/document/6600837?arnumber=6600837}
}
|
| Khargharia HS, Santana R, Shakya S, Ainslie R and Owusu G (2020), "Investigating RNNs for vehicle volume forecasting in service stations", In Proceedings of the 2020 IEEE Symposium Series on Computational Intelligence (SSCI-2020). Camberra, Australia , pp. 2625-2632. |
| Abstract: Accurate forecasting of customer demand can be critical for increasing operational efficiency and augmenting customer satisfaction, particularly in scenarios that involve multiple service units. In this paper, we focus on the problem of predicting the volume of vehicles in a network of gas stations and conduct an exhaustive investigation of different classes of recurrent neural networks for this problem. Particularly, we investigate the tradeoff between the accuracy and the overall complexity of sets of RNNs that employ varying number of models. We compare higher granularity models, where an RNN is learned from a particular dataset, to more general models sets, where a single neural network is learned from different but related datasets. Our results show that creating less specific models that integrate information from different related problems can decrease the computational cost of model learning with only a small decrease in terms of model accuracy. |
BibTeX:
@inproceedings{Khargharia_et_al:2020,
author = {Khargharia, Himadri Sikhar and Santana, Roberto and Shakya, Siddhartha and Ainslie, Russell and Owusu, Gilbert},
title = {Investigating RNNs for vehicle volume forecasting in service stations},
booktitle = {Proceedings of the 2020 IEEE Symposium Series on Computational Intelligence (SSCI-2020)},
year = {2020},
pages = {2625--2632},
url = {https://ieeexplore.ieee.org/document/9308368}
}
|
| Larrañaga P, Karshenas H, Bielza C and Santana R (2012), "A review on probabilistic graphical models in evolutionary computation", Journal of Heuristics. Vol. 18(5), pp. 795-819. |
| Abstract: Thanks to their inherent properties, probabilistic graphical models are one of the prime candidates for machine learning and decision making tasks especially in uncertain domains. Their capabilities, like representation, inference and learning, if used effectively, can greatly help to build intelligent systems that are able to act accordingly in different problem domains. Evolutionary algorithms is one such discipline that has employed probabilistic graphical models to improve the search for optimal solutions in complex problems. This paper shows how probabilistic graphical models have been used in evolutionary algorithms to improve their performance in solving complex problems. Specifically, we give a survey of probabilistic model building-based evolutionary algorithms, called estimation of distribution algorithms, and compare different methods for probabilistic modeling in these algorithms. |
BibTeX:
@article{Larranaga_et_al:2012,
author = {P. Larrañaga and H. Karshenas and C. Bielza and R. Santana},
title = {A review on probabilistic graphical models in evolutionary computation},
journal = {Journal of Heuristics},
year = {2012},
volume = {18},
number = {5},
pages = {795--819},
url = {http://dx.doi.org/10.1007/s10732-012-9208-4}
}
|
| Larrañaga P, Karshenas H, Bielza C and Santana R (2013), "A Review on Evolutionary Algorithms in Bayesian Network Learning and Inference Tasks", Information Sciences. Vol. 233(1), pp. 109-125. |
| Abstract: Thanks to their inherent properties, probabilistic graphical models are one of the prime candidates for machine learning and decision making tasks especially in uncertain domains. Their capabilities, like representation, inference and learning, if used effectively, can greatly help to build intelligent systems that are able to act accordingly in different problem domains. Bayesian networks are one of the most widely used class of these models. Some of the inference and learning tasks in Bayesian networks involve complex optimization problems that require the use of meta-heuristic algorithms. Evolutionary algorithms, as successful problem solvers, are promising candidates for this purpose. This paper reviews the application of evolutionary algorithms for solving some NP-hard optimization tasks in Bayesian network inference and learning. |
BibTeX:
@article{Larranaga_et_al:2013,
author = {P. Larrañaga and H. Karshenas and C. Bielza and R. Santana},
title = {A Review on Evolutionary Algorithms in Bayesian Network Learning and Inference Tasks},
journal = {Information Sciences},
year = {2013},
volume = {233},
number = {1},
pages = {109-125},
url = {https://www.sciencedirect.com/science/article/pii/S0020025513000443}
}
|
| Larrañaga P, Calvo B, Santana R, Bielza C, Galdiano J, Inza I, Lozano JA, Armañanzas R, Santafé G, Pérez A and Robles V (2006), "Machine learning in bioinformatics", Briefings in Bioinformatics. Vol. 7, pp. 86-112. |
| Abstract: This article reviews machine learning methods for bioinformatics. It presents modelling methods, such as supervised classification, clustering and probabilistic graphical models for knowledge discovery, as well as deterministic and stochastic heuristics for optimization. Applications in genomics, proteomics, systems biology, evolution and text mining are also shown. |
BibTeX:
@article{Larranhaga_et_al:2005,
author = {P. Larrañaga and B. Calvo and R. Santana and C. Bielza and J. Galdiano and I. Inza and J. A. Lozano and R. Armañanzas and G. Santafé and A. Pérez and V. Robles},
title = {Machine learning in bioinformatics},
journal = {Briefings in Bioinformatics},
year = {2006},
volume = {7},
pages = {86--112},
url = {http://dx.doi.org/10.1093/bib/bbk007}
}
|
| LaTorre A, Muelas S, Pena J, Santana R, Merchan-Perez A and Rodriguez J (2011), "A differential evolution algorithm for the detection of synaptic vesicles", In Evolutionary Computation (CEC), 2011 IEEE Congress on. , pp. 1687-1694. |
| Abstract: Neurotransmitters used by chemical synapses are stored in synaptic vesicles that accumulate in axon terminals. The number and position of these vesicles have been related to some important functional properties of the synapse. For this reason, an accurate mechanism for semi-automatically counting these small cellular structures will be of great help for neuroscientists. In this paper, we present a Differential Evolution algorithm that quantifies the number of synaptic vesicles in electron micrographs. The algorithm has been tested on several images that have been obtained from the somatosensory cortex of the rat and compared with some traditional approaches for detecting circular structures. Finally, the results have been validated by two independent expert anatomists. |
BibTeX:
@inproceedings{LaTorre_et_al:2011,
author = {LaTorre, A. and Muelas, S. and Pena, J.M. and Santana, R. and Merchan-Perez, A. and Rodriguez, J.R.},
title = {A differential evolution algorithm for the detection of synaptic vesicles},
booktitle = {Evolutionary Computation (CEC), 2011 IEEE Congress on},
year = {2011},
pages = {1687--1694},
url = {http://dx.doi.org/10.1109/CEC.2011.5949818}
}
|
| Lima RHR, Fontoura V, Pozo ATR and Santana R (2018), "Evolutionary Multi-Objective System Design: Theory and Applications. Computer and Information Science Series" , pp. 151-119. Chapman /& Hall/CRC. |
| Abstract: his chapter briefly introduces the main aspects of the Protein Folding Problem and presents a reviews of the related works. It presents an overview of the Multi-Objective optimization context and the Non-dominated Sorting Genetic Algorithm (NSGA-II) and Indicator-Based Algorithm (IBEA) algorithms are presented. The chapter presents the experimental benchmark and numerical results of the conducted experiments. It investigates the distance because more compact structures tend to have more hydrophobic contacts: as the lower the Euclidean distance between the amino acids is, the more compact the whole conformation will be. The prediction of protein structures has a wide range of important biotechnological and medical applications, e.g., design of new proteins and folds, structure-based drug design, and obtaining experimental structures from incomplete nuclear magnetic resonance data. The protein structures are the result of the so-called protein folding process in which the initially unfolded chain of amino acids is transformed into its final structure. |
BibTeX:
@inbook{Lima_et_al:2018,
author = {R. H. R. Lima and V. Fontoura and A. T. R. Pozo and R. Santana},
editor = {N. Nedjah and L. De-Macedo-Mourelle and H. Silverio-Lopes},
title = {Evolutionary Multi-Objective System Design: Theory and Applications. Computer and Information Science Series},
publisher = {Chapman /& Hall/CRC},
year = {2018},
pages = {151--119},
url = {https://www.taylorfrancis.com/chapters/edit/10.1201/9781315366845-8/multi-objective-approach-protein-structure-prediction-problem-ricardo-lima-vidal-fontoura-aurora-pozo-roberto-santana}
}
|
| Lima RHR, Pozo ATR and Santana R (2019), "Automatic design of convolutional neural networks using grammatical evolution", In 2019 8th Brazilian Conference on Intelligent Systems (BRACIS). , pp. 329-334. |
| Abstract: The use of Convolutional Neural Networks (CNNs) has been demonstrated to be a solid approach for solving many machine learning problems, such as image classification and natural language processing tasks. Usual CNN architectures are composed of many convolutions, pooling and fully connected layers, from which the networks also learn a suitable representation for the data being processed. The manual design of CNNs is a complex task due to the high number of possible parameter configurations. Recent studies about automatic design of CNNs have shown positive results. Since it can be expressed as a hyperparameter optimization problem, in this study we propose to explore the design of CNN architectures through the use of Grammatical Evolution (GE). GE is a grammar based approach where a grammar is used to define the CNN components and structural rules. We performed a set of experiments using two well-known image classification datasets, the MNIST and CIFAR-10. The obtained results show that the presented approach achieved competitive results, while maintaining relatively small architectures, when compared with similar state-of-the-art approaches. |
BibTeX:
@inproceedings{Lima_et_al:2019a,
author = {R. H. R. Lima and A. T. R. Pozo and R. Santana},
title = {Automatic design of convolutional neural networks using grammatical evolution},
booktitle = {2019 8th Brazilian Conference on Intelligent Systems (BRACIS)},
year = {2019},
pages = {329--334},
url = {https://ieeexplore.ieee.org/abstract/document/8923816}
}
|
| Lima RHR, Fontoura V, Pozo ATR, Mendiburu A and Santana R (2020), "A symmetric grammar approach for designing segmentation models", In 2020 IEEE Congress on Evolutionary Computation (CEC). , pp. 1-8. |
| Abstract: Image segmentation is a relevant problem in computer vision present in multiple application domains. One of the most used methods for image segmentation is U-net, a type of convolutional network with additional constraints in its architecture. Studies regarding the U-net usually rely on well-known architectures, which leads to a narrow exploration of the possibilities, and possibly impacting the performance. Genetic Programming approaches have become increasingly popular for designing neural networks due to studies where the generated models were able to achieve results comparable to humans. These approaches can evolve the structure at different levels of abstraction, reducing the need for a specialist. In this paper, we propose the use of Grammatical Evolution for evolving U-net architectures. We propose a mirror grammar, which is capable of generating a variety of flexible U-nets that better explores the search space. We show that the proposed grammar can capture the complex constraints that define the U-nets and achieve comparable results in terms of accuracy, on a benchmark of segmentation problems of varying difficulty. |
BibTeX:
@inproceedings{Lima_et_al:2020,
author = {R. H. R. Lima and V. Fontoura and A. T. R. Pozo and A. Mendiburu and R. Santana},
title = {A symmetric grammar approach for designing segmentation models},
booktitle = {2020 IEEE Congress on Evolutionary Computation (CEC)},
year = {2020},
pages = {1--8},
url = {https://ieeexplore.ieee.org/abstract/document/9185760}
}
|
| Lima RHR, Fontoura V, Pozo ATR, Mendiburu A and Santana R (2021), "Automatic Design of Deep Neural Networks Applied to Image Segmentation Problems", In Genetic Programming: 24th European Conference, EuroGP 2021, Held as Part of EvoStar 2021, Virtual Event, April 7--9, 2021, Proceedings. , pp. 98-112. |
| Abstract: A U-Net is a convolutional neural network mainly used for image segmentation domains such as medical image analysis. As other deep neural networks, the U-Net architecture influences the efficiency and accuracy of the network. We propose the use of a grammar-based evolutionary algorithm for the automatic design of deep neural networks for image segmentation tasks. The approach used is called Dynamic Structured Grammatical Evolution (DSGE), which employs a grammar to define the building blocks that are used to compose the networks, as well as the rules that help build them. We perform a set of experiments on the BSDS500 and ISBI12 datasets, designing networks tuned to image segmentation and edge detection. Subsequently, by using image similarity metrics, the results of our best performing networks are compared with the original U-Net. The results show that the proposed approach is able to design a network that is less complex in the number of trainable parameters, while also achieving slightly better results than the U-Net with a more consistent training. |
BibTeX:
@inproceedings{Lima_et_al:2021c,
author = {R. H. R. Lima and V. Fontoura and A. T. R. Pozo and A. Mendiburu and R. Santana},
title = {Automatic Design of Deep Neural Networks Applied to Image Segmentation Problems},
booktitle = {Genetic Programming: 24th European Conference, EuroGP 2021, Held as Part of EvoStar 2021, Virtual Event, April 7--9, 2021, Proceedings},
year = {2021},
pages = {98-112},
url = {https://www.springerprofessional.de/automatic-design-of-deep-neural-networks-applied-to-image-segmen/19000302}
}
|
| Lima RH, Magalhães D, Pozo A, Mendiburu A and Santana R (2022), "A grammar-based GP approach applied to the design of deep neural networks", Genetic Programming and Evolvable Machines. , pp. 1-26. Springer. |
| Abstract: Deep Learning has been very successful in automating the feature engineering process, widely applied for various tasks, such as speech recognition, classification, segmentation of images, time-series forecasting, among others. Deep neural networks (DNNs) incorporate the power to learn patterns through data, following an end-to-end fashion and expand the applicability in real world problems, since less pre-processing is necessary. With the fast growth in both scale and complexity, a new challenge has emerged regarding the design and configuration of DNNs. In this work, we present a study on applying an evolutionary grammar-based genetic programming algorithm (GP) as a unified approach to the design of DNNs. Evolutionary approaches have been growing in popularity for this subject as Neuroevolution is studied more. We validate our approach in three different applications: the design of Convolutional Neural Networks for image classification, Graph Neural Networks for text classification, and U-Nets for image segmentation. The results show that evolutionary grammar-based GP can efficiently generate different DNN architectures, adapted to each problem, employing choices that differ from what is usually seen in networks designed by hand. This approach has shown a lot of promise regarding the design of architectures, reaching competitive results with their counterparts. |
BibTeX:
@article{Lima_et_al:2022,
author = {Lima, Ricardo HR and Magalhães, Dimmy and Pozo, Aurora and Mendiburu, Alexander and Santana, Roberto},
title = {A grammar-based GP approach applied to the design of deep neural networks},
journal = {Genetic Programming and Evolvable Machines},
publisher = {Springer},
year = {2022},
pages = {1--26},
url = {https://link.springer.com/article/10.1007/s10710-022-09432-0}
}
|
| Lozada L and Santana R (2003), "UMDA dynamics for a class of parametric functions". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba, September, 2003. (ICIMAF 2003-239)
[BibTeX] |
BibTeX:
@techreport{Lozada_and_Santana:2003,
author = {L. Lozada and R. Santana},
title = {UMDA dynamics for a class of parametric functions},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {2003},
number = {ICIMAF 2003-239}
}
|
| Lozada-Chang L and Santana R (2011), "Univariate marginal distribution algorithm dynamics for a class of parametric functions with unitation constraints", Information Sciences. Vol. 181(11), pp. 2340-2355. |
| Abstract: In this paper, we introduce a mathematical model for analyzing the dynamics of the univariate marginal distribution algorithm (UMDA) for a class of parametric functions with isolated global optima. We prove a number of results that are used to model the evolution of UMDA probability distributions for this class of functions. We show that a theoretical analysis can assess the effect of the function parameters on the convergence and rate of convergence of UMDA. We also introduce for the first time a long string limit analysis of UMDA. Finally, we relate the results to ongoing research on the application of the estimation of distribution algorithms for problems with unitation constraints. |
BibTeX:
@article{Lozada_and_Santana:2011,
author = {L. Lozada-Chang and R. Santana},
title = {Univariate marginal distribution algorithm dynamics for a class of parametric functions with unitation constraints},
journal = {Information Sciences},
year = {2011},
volume = {181},
number = {11},
pages = {2340-2355},
url = {http://dx.doi.org/10.1016/j.ins.2011.01.024}
}
|
| Magalhães D, Pozo A and Santana R (2019), "An empirical comparison of distance/similarity measures for Natural Language Processing", In Anais do XVI Encontro Nacional de Inteligência Artificial e Computacional. Porto Alegre, RS, Brasil , pp. 717-728. SBC. |
| Abstract: Text Classification is one of the tasks of Natural Language Processing (NLP). In this area, Graph Convolutional Networks (GCN) has achieved values higher than CNN's and other related models. For GCN, the metric that defines the correlation between words in a vector space plays a crucial role in the classification because it determines the weight of the edges between two words (represented by nodes in the graph). In this study, we empirically investigated the impact of thirteen measures of distance/similarity. A representation was built for each document using word embedding from word2vec model. Also, a graph-based representation of five dataset was created for each measure analyzed, where each word is a node in the graph, and each edge is weighted by distance/similarity between words. Finally, each model was run in a simple graph neural network. The results show that, concerning text classification, there is no statistical difference between the analyzed metrics and the Graph Convolution Network. Even with the incorporation of external words or external knowledge, the results were similar to the methods without the incorporation of words. However, the results indicate that some distance metrics behave better than others in relation to context capture, with Euclidean distance reaching the best values or having statistical similarity with the best. |
BibTeX:
@inproceedings{Magalhaes_et_al:2019,
author = {Dimmy Magalhães and Aurora Pozo and Roberto Santana},
title = {An empirical comparison of distance/similarity measures for Natural Language Processing},
booktitle = {Anais do XVI Encontro Nacional de Inteligência Artificial e Computacional},
publisher = {SBC},
year = {2019},
pages = {717--728},
url = {https://sol.sbc.org.br/index.php/eniac/article/view/9328},
doi = {10.5753/eniac.2019.9328}
}
|
| Martins MS, Delgado MR, Santana R, Lüders R, Gonçalves RA and Almeida CPd (2016), "HMOBEDA: Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm", In Proceedings of the 2016 on Genetic and Evolutionary Computation Conference. , pp. 357-364. |
| Abstract: Probabilistic modeling of selected solutions and incorporation of local search methods are approaches that can notably improve the results of multi-objective evolutionary algorithms (MOEAs). In the past, these approaches have been jointly applied to multi-objective problems (MOPs) with excellent results. In this paper, we introduce for the first time a joint probabilistic modeling of (1) local search methods with (2) decision variables and (3) the objectives in a framework named HMOBEDA. The proposed approach is compared with six evolutionary methods (including a modified version of NSGA-III, adapted to solve combinatorial optimization) on instances of the multi-objective knapsack problem with 3, 4, and 5 objectives. Results show that HMOBEDA is a competitive approach. It outperforms the other methods according to the hypervolume indicator. |
BibTeX:
@inproceedings{Martins_et_al:2016,
author = {Martins, Marcella SR and Delgado, Myriam RBS and Santana, Roberto and Lüders, Ricardo and Gonçalves, Richard Aderbal and Almeida, Carolina Paula de},
title = {HMOBEDA: Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm},
booktitle = {Proceedings of the 2016 on Genetic and Evolutionary Computation Conference},
year = {2016},
pages = {357--364},
url = {https://dl.acm.org/doi/10.1145/2908812.2908826}
}
|
| Martins MSR, Delgado M, Lüders R, Santana R, Gonçalves RA and de Almeida CP (2017), "Probabilistic analysis of Pareto Front approximation for a hybrid multi-objective Bayesian estimation of distribution algorithm", In 2017 Brazilian Conference on Intelligent Systems (BRACIS). , pp. 384-389. |
| Abstract: https://ieeexplore.ieee.org/document/8247084 |
BibTeX:
@inproceedings{Martins_et_al:2017a,
author = {Martins, Marcella Scoczynski Ribeiro and Delgado, Myriam and Lüders, Ricardo and Santana, Roberto and Gonçalves, Richard A and de Almeida, Carolina P},
title = {Probabilistic analysis of Pareto Front approximation for a hybrid multi-objective Bayesian estimation of distribution algorithm},
booktitle = {2017 Brazilian Conference on Intelligent Systems (BRACIS)},
year = {2017},
pages = {384--389},
url = {https://ieeexplore.ieee.org/document/8247084}
}
|
| Martins MS, Delgado MR, Lüders R, Santana R, Gonçalves RA and Almeida CPd (2018), "Hybrid multi-objective Bayesian estimation of distribution algorithm: a comparative analysis for the multi-objective knapsack problem", Journal of Heuristics. Vol. 24(1), pp. 25-47. |
| Abstract: Nowadays, a number of metaheuristics have been developed for efficiently solving multi-objective optimization problems. Estimation of distribution algorithms are a special class of metaheuristic that intensively apply probabilistic modeling and, as well as local search methods, are widely used to make the search more efficient. In this paper, we apply a Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm (HMOBEDA) in multi and many objective scenarios by modeling the joint probability of decision variables, objectives, and the configuration parameters of an embedded local search (LS). We analyze the benefits of the online configuration of LS parameters by comparing the proposed approach with LS off-line versions using instances of the multi-objective knapsack problem with two to five and eight objectives. HMOBEDA is also compared with five advanced evolutionary methods using the same instances. Results show that HMOBEDA outperforms the other approaches including those with off-line configuration. HMOBEDA not only provides the best value for hypervolume indicator and IGD metric in most of the cases, but it also computes a very diverse solutions set close to the estimated Pareto front. |
BibTeX:
@article{Martins_et_al:2018,
author = {Martins, Marcella SR and Delgado, Myriam RBS and Lüders, Ricardo and Santana, Roberto and Gonçalves, Richard Aderbal and Almeida, Carolina Paula de},
title = {Hybrid multi-objective Bayesian estimation of distribution algorithm: a comparative analysis for the multi-objective knapsack problem},
journal = {Journal of Heuristics},
year = {2018},
volume = {24},
number = {1},
pages = {25-47},
url = {https://link.springer.com/article/10.1007/s10732-017-9356-7}
}
|
| Martins MS, Delgado MR, Lüders R, Santana R, Gonçalves RA and Almeida CPd (2018), "Exploring the probabilistic graphic model of a hybrid multi-objective Bayesian estimation of distribution algorithm", Applied Soft Computing. Vol. 73, pp. 328-343. |
| Abstract: The Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm (HMOBEDA) has shown to be very competitive for Many Objective Optimization Problems (MaOPs). The Probabilistic Graphic Model (PGM) of HMOBEDA expands the possibilities for exploration as it provides the joint probability of decision variables, objectives, and configuration parameters of an embedded local search. This work investigates different sampling mechanisms of HMOBEDA, applying the considered approaches to two different combinatorial MaOPs. Moreover, the paper provides a broad set of statistical analyses on its PGM model. These analyses have been carried out to evaluate how the interactions among variables, objectives and local search parameters are captured by the model and how information collected from different runs can be aggregated and explored in a Probabilistic Pareto Front. In experiments, two variants of HMOBEDA are compared with the original version, each one with a different set of evidences fixed during the sampling process. Results for instances of multi-objective knapsack problem with 2–5 and 8 objectives show that the best variant outperforms the original HMOBEDA in terms of convergence and diversity in the solution set. This best variant is then compared with five state-of-the-art evolutionary algorithms using the knapsack problem instances as well as a set of MNK-landscape instances with 2, 3, 5 and 8 objectives. HMOBEDA outperforms all of them. |
BibTeX:
@article{Martins_et_al:2018a,
author = {Martins, Marcella SR and Delgado, Myriam RBS and Lüders, Ricardo and Santana, Roberto and Gonçalves, Richard Aderbal and Almeida, Carolina Paula de},
title = {Exploring the probabilistic graphic model of a hybrid multi-objective Bayesian estimation of distribution algorithm},
journal = {Applied Soft Computing},
year = {2018},
volume = {73},
pages = {328-343},
url = {https://www.sciencedirect.com/science/article/pii/S1568494618305015}
}
|
| Martins MSR, El-Yafrani M, Santana R, Delgado M, Lueders R and Ahiod B (2018), "On the performance of multi-objective estimation of distribution algorithms for combinatorial problems", In IEEE Congress on Evolutionary Computation (CEC-2018). Rio de Janeiro, Brazil , pp. 1-8. |
| Abstract: Fitness landscape analysis investigates features with a high influence on the performance of optimization algorithms, aiming to take advantage of the addressed problem characteristics. In this work, a fitness landscape analysis using problem features is performed for a Multi-objective Bayesian Optimization Algorithm (mBOA) on instances of MNK-Iandscape problem for 2, 3, 5 and 8 objectives. We also compare the results of mBOA with those provided by NSGA-III through the analysis of their estimated runtime necessary to identify an approximation of the Pareto front. Moreover, in order to scrutinize the probabilistic graphic model obtained by mBOA, the Pareto front is examined according to a probabilistic view. The fitness landscape study shows that mBOA is moderately or loosely influenced by some problem features, according to a simple and a multiple linear regression model, which is being proposed to predict the algorithms performance in terms of the estimated runtime. Besides, we conclude that the analysis of the probabilistic graphic model produced at the end of evolution can be useful to understand the convergence and diversity performances of the proposed approach. |
BibTeX:
@inproceedings{Martins_et_al:2018b,
author = {M. S. R. Martins and M. El-Yafrani and R. Santana and M. Delgado and R. Lueders and B. Ahiod},
title = {On the performance of multi-objective estimation of distribution algorithms for combinatorial problems},
booktitle = {IEEE Congress on Evolutionary Computation (CEC-2018)},
year = {2018},
pages = {1--8},
url = {https://ieeexplore.ieee.org/document/8477970}
}
|
| Martins MS, Yafrani ME, Delgado M, Lüders R, Santana R, Siqueira HV, Akcay HG and Ahiod B (2021), "Analysis of Bayesian Network Learning Techniques for a Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm: a case study on MNK Landscape", Journal of Heuristics. Vol. 27(4), pp. 549-573. Springer. |
| Abstract: This work investigates different Bayesian network structure learning techniques by thoroughly studying several variants of Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm (HMOBEDA), applied to the MNK Landscape combinatorial problem. In the experiments, we evaluate the performance considering three different aspects: optimization abilities, robustness and learning efficiency. Results for instances of multi- and many-objective MNK-landscape show that, score-based structure learning algorithms appear to be the best choice. In particular, HMOBEDA𝑘2 was capable of producing results comparable with the other variants in terms of the runtime of convergence and the coverage of the final Pareto front, with the additional advantage of providing solutions that are less sensible to noise while the variability of the corresponding Bayesian network models is reduced. |
BibTeX:
@article{Martins_et_al:2021,
author = {Martins, Marcella SR and Yafrani, Mohamed El and Delgado, Myriam and Lüders, Ricardo and Santana, Roberto and Siqueira, Hugo V and Akcay, Huseyin G and Ahiod, Bela\id},
title = {Analysis of Bayesian Network Learning Techniques for a Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm: a case study on MNK Landscape},
journal = {Journal of Heuristics},
publisher = {Springer},
year = {2021},
volume = {27},
number = {4},
pages = {549--573},
url = {https://link.springer.com/article/10.1007/s10732-021-09469-x}
}
|
| Mei N, Sheikh U, Santana R and Soto D (2019), "How the brain encodes meaning: Comparing word embedding and computer vision models to predict fMRI data during visual word recognition", In Proceedings of the 2019 Conference on Cognitive Computational Neuroscience. Berlin, Germany , pp. 863-866. |
| Abstract: The brain representational spaces of conceptual knowledge remain unclear. We addressed this question in a functional MRI study in which 27 participants were required to either read visual words or think about the concepts that words represented. To examine the properties of the semantic representations in the brain, we tested different encoding models based on word embeddings models -FastText (Bojanowski, Grave, Joulin, & Mikolov, 2017), GloVe (Pennington, Socher, & Manning, 2014), word2vec (Mikolov, Sutskever, Chen, Corrado, & Dean, 2013)-, and, image vision models -VGG19 (Simonyan & Zisserman, 2014), MobileNetV2 (Howard et al., 2017), DenseNet121 (Huang, Liu, Van Der Maaten, & Weinberger, 2017)- fitted with the image referents of the words. These models were used to predict BOLD responses in putative substrates of the semantic network. We fitted and predicted the brain response using the feature representations extracted from the word embedding and computer vision models. Our results showed that computer vision models outperformed word embedding models in explaining brain responses during semantic processing tasks. Intriguingly, this pattern occurred independently of the task demand (reading vs thinking about the words). The results indicated that the abstract representations from the embedding layer of computer vision models provide a better semantic model of how the brain encodes word meaning. https://tinyurl.com/y5davcs6. |
BibTeX:
@inproceedings{Mei_et_al:2019,
author = {Mei, Ning and Sheikh, Usman and Santana, Roberto and Soto, David},
title = {How the brain encodes meaning: Comparing word embedding and computer vision models to predict fMRI data during visual word recognition},
booktitle = {Proceedings of the 2019 Conference on Cognitive Computational Neuroscience},
year = {2019},
pages = {863--866},
url = {https://ccneuro.org/2019/proceedings/0000863.pdf}
}
|
| Mei N, Santana R and Soto D (2021), "Informative neural representations of unseen objects during higher-order processing in human brains and deep artificial networks", bioRxiv. Cold Spring Harbor Laboratory. |
| Abstract: Despite advances in the neuroscience of visual consciousness over the last decades, we still lack a framework for understanding the scope of unconscious processing and how it relates to conscious experience. Previous research observed brain signatures of unconscious contents in visual cortex, but these have not been identified in a reliable manner, with low trial numbers and signal detection theoretic constraints not allowing to decisively discard conscious perception. Critically, the extent to which unconscious content is represented in high-level processing stages along the ventral visual stream and linked prefrontal areas remains unknown. Using a within-subject, high-precision, highly-sampled fMRI approach, we show that unconscious contents, even those associated with null sensitivity, can be reliably decoded from multivoxel patterns that are highly distributed along the ventral visual pathway and also involving prefrontal substrates. Notably, the neural representation in these areas generalised across conscious and unconscious visual processing states, placing constraints on prior findings that fronto-parietal substrates support the representation of conscious contents and suggesting revisions to models of consciousness such as the neuronal global workspace. We then provide a computational model simulation of visual information processing/representation in the absence of perceptual sensitivity by using feedforward convolutional neural networks trained to perform a similar visual task to the human observers. The work provides a novel framework for pinpointing the neural representation of unconscious knowledge across different task domains. |
BibTeX:
@article{Mei_et_al:2021,
author = {Mei, Ning and Santana, Roberto and Soto, David},
title = {Informative neural representations of unseen objects during higher-order processing in human brains and deep artificial networks},
journal = {bioRxiv},
publisher = {Cold Spring Harbor Laboratory},
year = {2021},
url = {https://www.biorxiv.org/content/10.1101/2021.01.12.426428v1.abstract}
}
|
| Mei N, Santana R and Soto D (2022), "Informative neural representations of unseen contents during higher-order processing in human brains and deep artificial networks", Nature Human Behaviour. Vol. 6(5), pp. 720-731. Nature Publishing Group. |
| Abstract: A framework to pinpoint the scope of unconscious processing is critical to improve models of visual consciousness. Previous research observed brain signatures of unconscious processing in visual cortex, but these were not reliably identified. Further, whether unconscious contents are represented in high-level stages of the ventral visual stream and linked parieto-frontal areas remains unknown. Using a within-subject, high-precision functional magnetic resonance imaging approach, we show that unconscious contents can be decoded from multi-voxel patterns that are highly distributed alongside the ventral visual pathway and also involving parieto-frontal substrates. Classifiers trained with multi-voxel patterns of conscious items generalized to predict the unconscious counterparts, indicating that their neural representations overlap. These findings suggest revisions to models of consciousness such as the neuronal global workspace. We then provide a computational simulation of visual processing/representation without perceptual sensitivity by using deep neural networks performing a similar visual task. The work provides a framework for pinpointing the representation of unconscious knowledge across different task domains. |
BibTeX:
@article{Mei_et_al:2022,
author = {Mei, Ning and Santana, Roberto and Soto, David},
title = {Informative neural representations of unseen contents during higher-order processing in human brains and deep artificial networks},
journal = {Nature Human Behaviour},
publisher = {Nature Publishing Group},
year = {2022},
volume = {6},
number = {5},
pages = {720--731},
url = {https://www.nature.com/articles/s41562-021-01274-7}
}
|
| Mendiburu A, Santana R, Bengoetxea E and Lozano J (2007), "A parallel framework for loopy belief propagation", In Proceedings of the Genetic and Evolutionary Computation Conference GECCO-2007. London, UK Vol. II, pp. 2843-2850. ACM Press. |
| Abstract: There are many innovative proposals introduced in the literature under the evolutionary computation field, from which estimation of distribution algorithms (EDAs) is one of them. Their main characteristic is the use of probabilistic models to represent the (in) dependencies between the variables of a concrete problem. Such probabilistic models have also been applied to the theoretical analysis of EDAs, providing a platform for the implementation of other optimization methods that can be incorporated into the EDA framework. Some of these methods, typically used for probabilistic inference, are belief propagation algorithms. In this paper we present a parallel approach for one of these inference-based algorithms, the loopy belief propagation algorithm for factor graphs. Our parallel implementation was designed to provide an algorithm that can be executed in clusters of computers or multiprocessors in order to reduce the total execution time. In addition, this framework was also designed as a flexible tool where many parameters, such as scheduling rules or stopping criteria, can be adjusted according to the requirements of each particular experiment and problem. |
BibTeX:
@inproceedings{Mendiburu_et_al:2007,
author = {A. Mendiburu and R. Santana and E. Bengoetxea and J. Lozano},
editor = {D. Thierens et al.},
title = {A parallel framework for loopy belief propagation},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference GECCO-2007},
publisher = {ACM Press},
year = {2007},
volume = {II},
pages = {2843--2850},
note = {Companion material},
url = {http://dl.acm.org/citation.cfm?id=1274084}
}
|
| Mendiburu A, Santana R and Lozano JA (2007), "Introducing belief propagation in estimation of distribution algorithms: A parallel framework". Research Report at: Department of Computer Science and Artificial Intelligence, University of the Basque Country., October, 2007. (EHU-KAT-IK-11/07) |
| Abstract: This paper incorporates Belief Propagation into an instance of Estimation of Distribution Algorithms called Estimation of Bayesian Networks Algorithm. Estimation of Bayesian Networks Algorithm learns a Bayesian network at each step. The objective of the proposed variation is to increase the search capabilities by extracting information of the, computationally costly to learn, Bayesian network. Belief Propagation applied to graphs with cycles, allows to find (with a low computational cost), in many scenarios, the point with the highest probability of a Bayesian network. We carry out some experiments to show how this modification can increase the potentialities of Estimation of Distribution Algorithms. Due to the computational time implied in the resolution of high dimensional optimization problems, we give a parallel version of the Belief Propagation algorithm for graphs with cycles and introduce it in a parallel framework for Estimation of Distribution Algorithms [13]. In addition we point out many ideas on how to incorporate Belief Propagation algorithms into Estimation Distribution Algorithms. |
BibTeX:
@techreport{Mendiburu_et_al:2007a,
author = {A. Mendiburu and R. Santana and J. A. Lozano},
title = {Introducing belief propagation in estimation of distribution algorithms: A parallel framework},
school = {Department of Computer Science and Artificial Intelligence, University of the Basque Country},
year = {2007},
number = {EHU-KAT-IK-11/07},
url = {https://www.researchgate.net/publication/228570851_Introducing_belief_propagation_in_estimation_of_distribution_algorithms_A_parallel_framework}
}
|
| Mendiburu A, Santana R and Lozano JA (2012), "Fast fitness improvements in Estimation of Distribution Algorithms using belief propagation", In Markov Networks in Evolutionary Computation. , pp. 141-155. Springer. |
| Abstract: Factor graphs can serve to represent Markov networks and Bayesian networks models. They can also be employed to implement efficient inference procedures such as belief propagation. In this paper we introduce a flexible implementation of belief propagation on factor graphs in the context of estimation of distribution algorithms (EDAs). By using a transformation from Bayesian networks to factor graphs, we show the way in which belief propagation can be inserted within the Estimation of Bayesian Networks Algorithm (EBNA). The objective of the proposed variation is to increase the search capabilities by extracting information of the, computationally costly to learn, Bayesian network. Belief Propagation applied to graphs with cycles allows to find (with a low computational cost), in many scenarios, the point with the highest probability of a Bayesian network. We carry out some experiments to show how this modification can increase the potentialities of Estimation of Distribution Algorithms. |
BibTeX:
@incollection{Mendiburu_et_al:2012,
author = {A. Mendiburu and R. Santana and J. A. Lozano},
editor = {R. Santana and S. Shakya},
title = {Fast fitness improvements in Estimation of Distribution Algorithms using belief propagation},
booktitle = {Markov Networks in Evolutionary Computation},
publisher = {Springer},
year = {2012},
pages = {141-155},
url = {http://dx.doi.org/10.1007/978-3-642-28900-2_9}
}
|
| Montenegro C, Lopez-Zorrilla A, Mikel-Olaso J, Santana R, Justo R, Lozano JA and Torres MI (2019), "A Dialogue-Act Taxonomy for a Virtual Coach Designed to Improve the Life of Elderly", Multimodal Technologies and Interaction. Vol. 3(3), pp. 52. MDPI. |
| Abstract: This paper presents a dialogue act taxonomy designed for the development of a conversational agent for elderly. The main goal of this conversational agent is to improve life quality of the user by means of coaching sessions in different topics. In contrast to other approaches such as task-oriented dialogue systems and chit-chat implementations, the agent should display a pro-active attitude, driving the conversation to reach a number of diverse coaching goals. Therefore, the main characteristic of the introduced dialogue act taxonomy is its capacity for supporting a communication based on the GROW model for coaching. In addition, the taxonomy has a hierarchical structure between the tags and it is multimodal. We use the taxonomy to annotate a Spanish dialogue corpus collected from a group of elder people. We also present a preliminary examination of the annotated corpus and discuss on the multiple possibilities it presents for further research. |
BibTeX:
@article{Montenegro_et_al:2019,
author = {C. Montenegro and A. Lopez-Zorrilla and J. Mikel-Olaso and R. Santana and R. Justo and J. A. Lozano and M. I. Torres},
title = {A Dialogue-Act Taxonomy for a Virtual Coach Designed to Improve the Life of Elderly},
journal = {Multimodal Technologies and Interaction},
publisher = {MDPI},
year = {2019},
volume = {3},
number = {3},
pages = {52},
url = {https://www.mdpi.com/2414-4088/3/3/52}
}
|
| Montenegro C, Santana R and Lozano JA (2019), "Data generation approaches for topic classification in multilingual spoken dialog system", In Proceedings of the 12th Conference on PErvasive Technologies Related to Assistive Environments Conference (PETRA-19). Rhodes, Greece , pp. 211-217. ACM. |
| Abstract: The conception of spoken-dialog systems (SDS) usually faces the problem of extending or adapting the system to multiple languages. This implies the creation of modules specifically for the new languages, which is a time consuming process. In this paper, we propose two methods to reduce the time needed to extend the SDS to other languages. Our methods are particularly oriented to the topic classification and semantic tagging tasks and we evaluate their effectiveness on topic classification for three languages: English, Spanish, French. |
BibTeX:
@inproceedings{Montenegro_et_al:2019,
author = {C. Montenegro and R. Santana and J. A. Lozano},
title = {Data generation approaches for topic classification in multilingual spoken dialog system},
booktitle = {Proceedings of the 12th Conference on PErvasive Technologies Related to Assistive Environments Conference (PETRA-19)},
publisher = {ACM},
year = {2019},
pages = {211-217},
url = {https://dl.acm.org/doi/10.1145/3316782.3316792}
}
|
| Montenegro C, Santana R and Lozano JA (2020), "Transfer learning in hierarchical dialogue topic classification with neural networks", In Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN-2020). Glasgow, UK , pp. 1-8. |
| Abstract: Knowledge transfer between tasks can significantly improve the efficiency of machine learning algorithms. In supervised natural language understanding problems, this sort of improvement is critical since the availability of labelled data is usually scarce. In this paper we address the question of transfer learning between related topic classification tasks. A characteristic of our problem is that the tasks have a hierarchical relationship. Therefore, we introduce and validate how to implement the transfer exploiting this hierarchical structure. Our results for a real-world topic classification task show that the transfer can produce improvements in the behavior of the classifiers for some particular problems. |
BibTeX:
@inproceedings{Montenegro_et_al:2020,
author = {C. Montenegro and R. Santana and J. A. Lozano},
title = {Transfer learning in hierarchical dialogue topic classification with neural networks},
booktitle = {Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN-2020)},
year = {2020},
pages = {1--8},
url = {https://ieeexplore.ieee.org/abstract/document/9206680}
}
|
| Montenegro C, Santana R and Lozano JA (2021), "Analysis of the sensitivity of the End-Of-Turn Detection task to errors generated by the Automatic Speech Recognition process", Engineering Applications of Artificial Intelligence. Vol. 100, pp. 104189. Elsevier. |
| Abstract: An End-Of-Turn Detection Module (EOTD-M) is an essential component of automatic Spoken Dialogue Systems. The capability of correctly detecting whether a user’s utterance has ended or not improves the accuracy in interpreting the meaning of the message and decreases the latency in the answer. Usually, in dialogue systems, an EOTD-M is coupled with an Automatic Speech Recognition Module (ASR-M) to transmit complete utterances to the Natural Language Understanding unit. Mistakes in the ASR-M transcription can have a strong effect on the performance of the EOTD-M. The actual extent of this effect depends on the particular combination of ASR-M transcription errors and the sentence featurization techniques implemented as part of the EOTD-M. In this paper we investigate this important relationship for an EOTD-M based on semantic information and particular characteristics of the speakers (speech profiles). We introduce an Automatic Speech Recognition Simulator (ASR-SIM) that models different types of semantic mistakes in the ASR-M transcription as well as different speech profiles. We use the simulator to evaluate the sensitivity to ASR-M mistakes of a Long Short-Term Memory network classifier trained in EOTD with different featurization techniques. Our experiments reveal the different ways in which the performance of the model is influenced by the ASR-M errors. We corroborate that not only is the ASR-SIM useful to estimate the performance of an EOTD-M in customized noisy scenarios, but it can also be used to generate training datasets with the expected error rates of real working conditions, which leads to better performance. |
BibTeX:
@article{Montenegro_et_al:2021,
author = {C. Montenegro and R. Santana and J. A. Lozano},
title = {Analysis of the sensitivity of the End-Of-Turn Detection task to errors generated by the Automatic Speech Recognition process},
journal = {Engineering Applications of Artificial Intelligence},
publisher = {Elsevier},
year = {2021},
volume = {100},
pages = {104189},
url = {https://www.sciencedirect.com/science/article/pii/S0952197621000361}
}
|
| Murua M, Suárez A, de-Lacalle NL, Santana R and Wretland A (2018), "Feature extraction based prediction of tool wear of Inconel 718 in face turning", Insight. Non-Destructive Testing and Condition Monitoring. Vol. 60(8), pp. 1-8. The British Institute of Non-Destructive Testing. |
| Abstract: Tool wear is a recurring topic in the cutting field, so obtaining knowledge about the tool wear process and the capability of predicting tool wear is of special importance. Cutting processes can be optimised with predictive models that are able to forecast tool wear with a suitable level of accuracy. This research focuses on the application of some regression approaches, based on machine learning techniques, to a face-turning process for Inconel 718. To begin with, feature extraction of the cutting forces is considered, to generate regression models. Subsequently, the regression models are improved with a reduced set of features obtained by computing the feature importance. The results provide evidence that the gradient-boosting regressor allows an increment in the wear prediction accuracy and the random forest regressor has the capability of detecting relevant features that characterise the turning process. They also reveal higher accuracy in predicting tool wear under high-pressure cooling as opposed to conventional lubrication. |
BibTeX:
@article{Murua_et_al:2018,
author = {M. Murua and A. Suárez and N. López-de-Lacalle and R. Santana and A. Wretland},
title = {Feature extraction based prediction of tool wear of Inconel 718 in face turning},
journal = {Insight. Non-Destructive Testing and Condition Monitoring},
publisher = {The British Institute of Non-Destructive Testing},
year = {2018},
volume = {60},
number = {8},
pages = {1--8},
url = {https://www.ingentaconnect.com/content/bindt/insight/2018/00000060/00000008/art00006}
}
|
| Murua M, Suárez A, Galar D, Santana R and Wretland A (2020), "Tool-Path Problem in Direct Energy Deposition Metal-Additive Manufacturing: Sequence Strategy Generation.", IEEE Access. Vol. 8(9093820), pp. 91574-91585. IEEE Press. |
| Abstract: The tool-path problem has been extensively studied in manufacturing technologies, as it has a considerable impact on production time. Additive manufacturing is one of these technologies; it takes time to fabricate parts, so the selection of optimal tool-paths is critical. This research analyzes the tool-path problem in the direct energy deposition technology; it introduces the main processes, and analyzes the characteristics of tool-path problem. It explains the approaches applied in the literature to solve the problem; as these are mainly geometric approximations, they are far from optimal. Based on this analysis, this paper introduces a mathematical framework for direct energy deposition and a novel problem called sequence strategy generation. Finally, it solves the problem using a benchmark for several different parts. The results reveal that the approach can be applied to parts with different characteristics, and the solution to the sequence strategy problem can be used to generate tool-paths. |
BibTeX:
@article{Murua_et_al:2020,
author = {M. Murua and A. Suárez and D. Galar and R. Santana and A. Wretland},
title = {Tool-Path Problem in Direct Energy Deposition Metal-Additive Manufacturing: Sequence Strategy Generation.},
journal = {IEEE Access},
publisher = {IEEE Press},
year = {2020},
volume = {8},
number = {9093820},
pages = {91574-91585},
url = {https://ieeexplore.ieee.org/document/9093820}
}
|
| Murua M, Galar D and Santana R (2020), "Adaptation of a Branching Algorithm to Solve the Multi-Objective Hamiltonian Cycle Problem", In Operations Research Proceedings 2019. , pp. 231-237. Springer. |
| Abstract: The Hamiltonian cycle problem (HCP) consists of finding a cycle of length N in an N-vertices graph. In this investigation, a graph G is considered with an associated set of matrices, in which each cell in the matrix corresponds to the weight of an arc. Thus, a multi-objective variant of the HCP is addressed and a Pareto set of solutions that minimizes the weights of the arcs for each objective is computed. To solve the HCP problem, the Branch-and-Fix algorithm is employed, a specific branching algorithm that uses the embedding of the problem in a particular stochastic process. To address the multi-objective HCP, the Branch-and-Fix algorithm is extended by computing different Hamiltonian cycles and fathoming the branches of the tree at earlier stages. The introduced anytime algorithm can produce a valid solution at any time of the execution, improving the quality of the Pareto Set as time increases. |
BibTeX:
@incollection{Murua_et_al:2020a,
author = {Murua, Maialen and Galar, Diego and Santana, Roberto},
title = {Adaptation of a Branching Algorithm to Solve the Multi-Objective Hamiltonian Cycle Problem},
booktitle = {Operations Research Proceedings 2019},
publisher = {Springer},
year = {2020},
pages = {231--237},
url = {https://link.springer.com/chapter/10.1007/978-3-030-48439-2_28}
}
|
| Murua M, Galar D and Santana R (2022), "Solving the multi-objective Hamiltonian cycle problem using a Branch-and-Fix based algorithm", Journal of Computational Science. Vol. 60, pp. 101578. Elsevier. |
| Abstract: The Hamiltonian cycle problem consists of finding a cycle in a given graph that passes through every single vertex exactly once, or determining that this cannot be achieved. In this investigation, a graph is considered with an associated set of matrices. The entries of each of the matrix correspond to a different weight of an arc. A multi-objective Hamiltonian cycle problem is addressed here by computing a Pareto set of solutions that minimize the sum of the weights of the arcs for each objective. Our heuristic approach extends the Branch-and-Fix algorithm, an exact method that embeds the problem in a stochastic process. To measure the efficiency of the proposed algorithm, we compare it with a multi-objective genetic algorithm in graphs of a different number of vertices and density. The results show that the density of the graphs is critical when solving the problem. The multi-objective genetic algorithm performs better (quality of the Pareto sets) than the proposed approach in random graphs with high density; however, in these graphs it is easier to find Hamiltonian cycles, and they are closer to the multi-objective traveling salesman problem. The results reveal that, in a challenging benchmark of Hamiltonian graphs with low density, the proposed approach significantly outperforms the multi-objective genetic algorithm. |
BibTeX:
@article{Murua_et_al:2022,
author = {Murua, M and Galar, D and Santana, R},
title = {Solving the multi-objective Hamiltonian cycle problem using a Branch-and-Fix based algorithm},
journal = {Journal of Computational Science},
publisher = {Elsevier},
year = {2022},
volume = {60},
pages = {101578},
url = {https://www.sciencedirect.com/science/article/pii/S1877750322000151}
}
|
| Ochoa A, Soto MR, Santana R, Madera J and Jorge N (1999), "The Factorized Distribution Algorithm and the Junction Tree: A Learning Perspective", In Proceedings of the Second Symposium on Artificial Intelligence (CIMAF-99). Havana, Cuba, March, 1999. , pp. 368-377. Editorial Academia. Havana, Cuba. |
| Abstract: This paper extends the FDA - the Factorized Distribution Algorithm - with a structural learning component. The FDA has been extensively investigated for the optimization of additively decomposed discrete functions (ADFs). Now, we are able to deal with more general problems, which are solved by FDA in a blackbox optimization scenario. The key point is the construction of the Junction Tree, which is placed at the centre of the algorithm. Learning the Junction Tree directly from the data is a process that is accomplished by making independency tests of as lower as possible order. The proposed algorithm belongs to the class of Estimation Distribution Algorithms and represents an interesting alternative to approach the Linkage Problem in Genetic Algorithms. |
BibTeX:
@inproceedings{Ochoa_et_al:1999,
author = {A. Ochoa and M. R. Soto and R. Santana and J. Madera and N. Jorge},
editor = {A. Ochoa and M. R. Soto and R. Santana},
title = {The Factorized Distribution Algorithm and the Junction Tree: A Learning Perspective},
booktitle = {Proceedings of the Second Symposium on Artificial Intelligence (CIMAF-99)},
publisher = {Editorial Academia. Havana, Cuba},
year = {1999},
pages = {368--377},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/1999/deo012.pdf}
}
|
| Rodríguez-Ojea LM, Santana R and Soto MR (2003), "Uso de algorithmos evolutivos con estimación de distribuciones en la solución de problemas enteros", In Proceedings of the International Conference INFORMATICA-2004. La Habana, Cuba |
| Abstract: El presente trabajo presenta un análisis del empleo de EDAs basados en modelos gráficos simplemente conectados en la solución de problemas con codificación entera. Se realiza un estudio del desempeño de los EDAs implementados y un análisis empírico del aumento de la complejidad de los referidos algoritmos como consecuencia del paso de la codificación binaria a la codificación entera. |
BibTeX:
@inproceedings{Ojea_et_al:2004,
author = {L. M. Rodríguez-Ojea and R. Santana and M. R. Soto},
title = {Uso de algorithmos evolutivos con estimación de distribuciones en la solución de problemas enteros},
booktitle = {Proceedings of the International Conference INFORMATICA-2004},
year = {2003}
}
|
| Olaso JM, Vázquez A, Ben Letaifa L, De Velasco M, Mtibaa A, Hmani MA, Petrovska-Delacrétaz D, Chollet G, Montenegro C, López-Zorrilla A, Justo R, Santana R and others (2021), "The EMPATHIC Virtual Coach: a demo", In Proceedings of the 2021 International Conference on Multimodal Interaction. , pp. 848-851. |
| Abstract: The main objective of the EMPATHIC project has been the design and development of a virtual coach to engage the healthy-senior user and to enhance well-being through awareness of personal status. The EMPATHIC approach addresses this objective through multimodal interactions supported by the GROW coaching model. The paper summarizes the main components of the EMPATHIC Virtual Coach (EMPATHIC-VC) and introduces a demonstration of the coaching sessions in selected scenarios. |
BibTeX:
@inproceedings{Olaso_et_al:2021,
author = {Olaso, Javier M and Vázquez, Alain and Ben Letaifa, Leila and De Velasco, Mikel and Mtibaa, Aymen and Hmani, Mohamed Amine and Petrovska-Delacrétaz, Dijana and Chollet, Gérard and Montenegro, César and López-Zorrilla, Asier and Justo, Raquel and Santana, Roberto and others},
title = {The EMPATHIC Virtual Coach: a demo},
booktitle = {Proceedings of the 2021 International Conference on Multimodal Interaction},
year = {2021},
pages = {848--851},
url = {https://dl.acm.org/doi/abs/10.1145/3462244.3481574}
}
|
| Pereira FB, Machado P, Costa E, Cardoso A, Ochoa A, Santana R and Soto MR (2000), "Too busy to learn", In Proceedings of the 2000 Congress on Evolutionary Computation CEC-2000. La Jolla Marriott Hotel La Jolla, California, USA, July, 2000. , pp. 720-727. IEEE Press. |
| Abstract: The goal of this research is to analyze how individual learning helps an evolutionary algorithm in its search for best candidates for the Busy Beaver problem. To study this interaction two learning models, implemented as local search procedures, are proposed. Experimental results show that, in highly irregular and very prone to premature convergence search spaces, local search methods are not an effective help to evolution. In addition, one interesting effect related to learning is reported. When the mutation rate is too high, learning acts as a repair, reintroducing some useful information that was lost |
BibTeX:
@inproceedings{Pereira_et_al:2000,
author = {Francisco B. Pereira and Penousal Machado and Ernesto Costa and Amílcar Cardoso and Alberto Ochoa and Roberto Santana and Marta Rosa Soto},
title = {Too busy to learn},
booktitle = {Proceedings of the 2000 Congress on Evolutionary Computation CEC-2000},
publisher = {IEEE Press},
year = {2000},
pages = {720--727},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/2000/CEC2000.pdf}
}
|
| Pereira FB, Machado P, Costa E, Cardoso A, Ochoa A, Santana R and Soto MR (2000), "Too Busy to Learn", In Colectânea de Comunicacões. Instituto Politécnico de Coimbra , pp. 699-712. Ediliber, Lda.. |
| Abstract: The goal of this research is to analyze how individual learning helps an evolutionary algorithm in its search for best candidates for the Busy Beaver problem. To study this interaction two learning models, implemented as local search procedures, are proposed. Experimental results show that, in highly irregular and very prone to premature convergence search spaces, local search methods are not an effective help to evolution. In addition, one interesting effect related to learning is reported. When the mutation rate is too high, learning acts as a repair, reintroducing some useful information that was lost |
BibTeX:
@incollection{Pereira_et_al:2000a,
author = {Francisco B. Pereira and Penousal Machado and Ernesto Costa and Amílcar Cardoso and Alberto Ochoa and Roberto Santana and Marta Rosa Soto},
title = {Too Busy to Learn},
booktitle = {Colectânea de Comunicacões},
publisher = {Ediliber, Lda.},
year = {2000},
pages = {699--712},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/2000/CEC2000.pdf}
}
|
| Picek S, McKay RI, Santana R and Gedeon TD (2015), "Fighting the Symmetries: The Structure of Cryptographic Boolean Function Spaces", In Proceedings of the Companion Publication of the 2015 on Genetic and Evolutionary Computation Conference. Madrid, Spain , pp. 457-464. |
| Abstract: We explore the problem space of maximum nonlinearity problems for balanced Boolean functions, examining the symmetry structure and fitness landscapes in the most common (bit string) representation. We present theoretical analyses of well understood aspects, together with detailed enumeration of the 4-bit problem, sampling of the 6-bit problem based on known optima, and sampling of the 8-bit problem based on its fittest known solutions. We show that these problems have many more symmetries than is generally noted, with implications for crossover and for distributional methods. We explore the large-scale plateau structure of the problem, with similar implications for local search. We show that symmetries yield additional information that may yield more effective search methods. |
BibTeX:
@inproceedings{Picek_et_al:2015,
author = {S. Picek and R. I. McKay and R. Santana and T. D. Gedeon},
title = {Fighting the Symmetries: The Structure of Cryptographic Boolean Function Spaces},
booktitle = {Proceedings of the Companion Publication of the 2015 on Genetic and Evolutionary Computation Conference},
year = {2015},
pages = {457--464},
url = {https://dl.acm.org/doi/10.1145/2739480.2754739}
}
|
| Picek S, Santana R and Jakobovic D (2016), "Maximal nonlinearity in balanced boolean functions with even number of inputs, revisited", In 2016 IEEE Congress on Evolutionary Computation (CEC). , pp. 3222-3229. |
| Abstract: The problem of obtaining maximal nonlinearity in Boolean functions is well researched, both from the cryptographic and the evolutionary computation side. However, the results are still not conclusive enough to be able to show how good a heuristic approach is when tackling this problem. In this paper, we investigate how to obtain the maximal possible nonlinearity in balanced Boolean functions, but we also analyze how difficult is the problem itself. In order to do so, we conduct experiments with Estimation of distribution algorithms as well as the fitness landscape analysis and the deception analysis. Our results indicate that the first difficulties arise from the inappropriate fitness function and representation of solutions coupled with a huge search space. The fitness landscape analysis does not reveal any significant differences that could justify the assumed jump in problem difficulty when going from Boolean functions with 6 inputs to those with 8 inputs. Finally, we show that this problem is not order-1 deceptive. |
BibTeX:
@inproceedings{Picek_et_al:2016,
author = {Picek, Stjepan and Santana, Roberto and Jakobovic, Domagoj},
title = {Maximal nonlinearity in balanced boolean functions with even number of inputs, revisited},
booktitle = {2016 IEEE Congress on Evolutionary Computation (CEC)},
year = {2016},
pages = {3222--3229},
url = {https://ieeexplore.ieee.org/document/7744197}
}
|
| de León EP and Santana R (1997), "A genetic algorithm and a local search procedure for a Hamiltonian path problem", In Memorias del Encuentro Latino Iberoamericano de Optimización, I ELIO. Congreso Chileno de Investigación Operativa OPTIMA 97.. University of Conception, Chile, November 3-6, 1997. , pp. 226-232. |
| Abstract: This paper deals with Hávels conjecture that asserts that all bipartite graph formed with the middle levels of n-dimensional cube is Hamiltonian, when n is odd. This means that at least one Hamiltonian cycle should be found in the graph. We use a circular representation of the vertices of n-dimensional cube which make it possible to introduce two group actions in order to reduce the problem to find out a Hamiltonian path in multi - level quotient graphs. We report on a hybrid genetic algorithm for the Hávels conjecture problem. We present interesting results which show that this GA approach gives optimal solutions in multi- level quotient graph. We use bitstring representation, an evolutionary fitness function, restricted edge crossover operator, intelligent mutation, seeding and adaptive mutation rate. Three Hamiltonian path are constructed with this method. |
BibTeX:
@inproceedings{Ponce_and_Santana:1997,
author = {E. Ponce de León and R. Santana},
title = {A genetic algorithm and a local search procedure for a Hamiltonian path problem},
booktitle = {Memorias del Encuentro Latino Iberoamericano de Optimización, I ELIO. Congreso Chileno de Investigación Operativa OPTIMA 97.},
year = {1997},
pages = {226-232}
}
|
| de León EP and Santana R (1999), "A hybrid genetic algorithm for a Hamiltonian path problem", Revista Investigación Operacional. Vol. 20(1), pp. 20-29. Universidad de la Habana. |
| Abstract: This paper deals with Hável and Erdös conjecture that asserts that all bipartite graph formed with the middle levels of n-dimensional cube is Hamiltonian, when n is odd. This means that at least one Hamiltonian cycle should be found in the graph. We use a circular representation of the vertices of n-dimensional cube which makes it possible to introduce two group actions in order to reduce the problem to find out a Hamiltonian path in multi - level quotient graphs. We report on a hybrid genetic algorithm for the Hável and Erdös conjecture problem. We present interesting results which show that this GA approach gives optimal solutions in multi-level quotient graph. We use bitstring representation, an evolutionary fitness function, restricted edge crossover operator, intelligent mutation, seeding and adaptive mutation rate. Three Hamiltonian paths are constructed with this method. |
BibTeX:
@article{Ponce_and_Santana:1999,
author = {E. Ponce de León and R. Santana},
title = {A hybrid genetic algorithm for a Hamiltonian path problem},
journal = {Revista Investigación Operacional},
publisher = {Universidad de la Habana},
year = {1999},
volume = {20},
number = {1},
pages = {20--29}
}
|
| de León EP, Ochoa A and Santana R (1996), "A genetic algorithm for a Hamiltonian path problem". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba, September, 1996. (ICIMAF 96-01, CENIA 95-01)
[BibTeX] |
BibTeX:
@techreport{Ponce_et_al:1996,
author = {E. Ponce de León and A. Ochoa and R. Santana},
title = {A genetic algorithm for a Hamiltonian path problem},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {1996},
number = {ICIMAF 96-01, CENIA 95-01}
}
|
| de León EP, Santana R and Maldonado F (1997), "Algoritmos Genéticos y el problema del teñido de telas en la Industria Textil", In Proceedings of the First Symposium on Artificial Intelligence (CIMAF-97). Havana, Cuba, March, 1997. , pp. 18-28. Editora de la Academia de Ciencias de Cuba. |
| Abstract: La importancia práctica de la búsqueda de una secuencia de colores óptima en el proceso de teñido de prendas es bien conocido en el ambiente textil. Nosotros presentamos un enfoque a este problema utilizando Algoritmos Genéticos (AGs). Ellos han mostrado eficientes resultados en multiples problemas del mundo real. Su fuente de origen fue la observación, por parte de los investigadores de las ciencias de la computación, de los procesos de optimización que ocurren en la naturaleza. La modelación matem/'atica del problema nos conduce a identificarlo como el problema del viajante de comercio asimétrico. Se diseña una familia de AGs para el problema en cuestión. Un operador de entrecruzamiento es introducido. La función de adaptabilidad aprovecha información en los dos sentidos de la secuencia de colores a evaluar. Se presentan diferentes formas de aprovechar el conocimiento heurístico del problema. Resultados de este primer acercamiento al problema con AG son discutidos. |
BibTeX:
@inproceedings{Ponce_et_al:1997,
author = {E. Ponce de León and R. Santana and F. Maldonado},
title = {Algoritmos Genéticos y el problema del teñido de telas en la Industria Textil},
booktitle = {Proceedings of the First Symposium on Artificial Intelligence (CIMAF-97)},
publisher = {Editora de la Academia de Ciencias de Cuba},
year = {1997},
pages = {18-28}
}
|
| de León EP, Santana R, Brito I and Maldonado F (1997), "Análisis de una familia de algoritmos genéticos para un problema de secuencias en el teñido de telas", In Memorias del Encuentro Latino Iberoamericano de Optimización, I ELIO. Congreso Chileno de Investigación Operativa OPTIMA 97.. University of Conception, Chile, November 3-6, 1997.
[BibTeX] |
BibTeX:
@inproceedings{Ponce_et_al:1997a,
author = {E. Ponce de León and R. Santana and I. Brito and F. Maldonado},
title = {Análisis de una familia de algoritmos genéticos para un problema de secuencias en el teñido de telas},
booktitle = {Memorias del Encuentro Latino Iberoamericano de Optimización, I ELIO. Congreso Chileno de Investigación Operativa OPTIMA 97.},
year = {1997},
note = {In Spanish}
}
|
| de León EP, Santana R and Ochoa A (1997), "A genetic algorithm for a Hamiltonian path problem: Mutation - crossover interaction", In Proceedings of the 13th ISPE/IEE International Conference on CAD/CAM Robotics and Factories of the Future 97. Universidad Tecnológica de Pereira, Colombia, December, 1997. , pp. 1001-1006. |
| Abstract: The GA method is especially useful in cases when the hypersurface, in which the optimum is searched, is of a high dimension and has many local optima. The complexity of such problems renders an exhaustive search through the space (using, for example, a grid search) useless. Due to local optima, there is a danger that direct optimization methods stop far away from the global optimum. |
BibTeX:
@inproceedings{Ponce_et_al:1997b,
author = {E. Ponce de León and R. Santana and A. Ochoa},
title = {A genetic algorithm for a Hamiltonian path problem: Mutation - crossover interaction},
booktitle = {Proceedings of the 13th ISPE/IEE International Conference on CAD/CAM Robotics and Factories of the Future 97},
year = {1997},
pages = {1001-1006}
}
|
| de León EP, Santana R and Maldonado F (1997), "Algoritmos Genéticos y el problema del teñido de telas en la Industria Textil", Revista Latinoamericana de Tecnología Textil. IPN, Meexico Vol. 1(8), pp. 49-58. |
| Abstract: La importancia práctica de la búsqueda de una secuencia de colores óptima en el proceso de teñido de prendas es bien conocido en el ambiente textil. Nosotros presentamos un enfoque a este problema utilizando Algoritmos Genéticos (AGs). Ellos han mostrado eficientes resultados en multiples problemas del mundo real. Su fuente de origen fue la observación, por parte de los investigadores de las ciencias de la computación, de los procesos de optimización que ocurren en la naturaleza. La modelación matem/'atica del problema nos conduce a identificarlo como el problema del viajante de comercio asimétrico. Se diseña una familia de AGs para el problema en cuestión. Un operador de entrecruzamiento es introducido. La función de adaptabilidad aprovecha información en los dos sentidos de la secuencia de colores a evaluar. Se presentan diferentes formas de aprovechar el conocimiento heurístico del problema. Resultados de este primer acercamiento al problema con AG son discutidos. |
BibTeX:
@article{Ponce_et_al:1997c,
author = {E. Ponce de León and R. Santana and F. Maldonado},
title = {Algoritmos Genéticos y el problema del teñido de telas en la Industria Textil},
journal = {Revista Latinoamericana de Tecnología Textil},
year = {1997},
volume = {1},
number = {8},
pages = {49--58},
note = {In Spanish}
}
|
| de León EP, Ochoa A and Santana R (1997), "A Genetic Algorithm for a Hamiltonian Path Problem", In Proceedings of the X International Conference on Industrial and Engineering Applications of AI and Expert Systems. Atlanta. USA , pp. 13-19. |
| Abstract: The genetic algorithm (GA) is one of the stochastic search techniques with application to a wide variety of combinatorial optimization problems. A conjecture of I. Havel, also attributed to P. Erdos, asserts that the simple graph G(k+1) (k>0), whose vertices are the subsets of cardinalities k and k+1 of the set 0,...,2k and whose adjacency is given by subset inclusion, is Hamiltonian. The search of a Hamiltonian cycle in this graph is reduced to find a Hamiltonian path in the multi-level graph. In this paper we describe a GA approach to this conjecture. We present theoretical and computational results which show that this GA approach finds the optimal solutions when we search path for each level of the graph. We introduce an evolutive fitness function, and discuss the impact of using non standard crossover operators. Seeding and adaptive mutation rate are used. Hamiltonian cycle in G(6) and G(7) are constructed. |
BibTeX:
@inproceedings{Ponce_et_al:1997d,
author = {E. Ponce de León and A. Ochoa and R. Santana},
title = {A Genetic Algorithm for a Hamiltonian Path Problem},
booktitle = {Proceedings of the X International Conference on Industrial and Engineering Applications of AI and Expert Systems},
year = {1997},
pages = {13--19},
url = {http://books.google.es/books?hl=en&lr=&id=48FMFE2VwPkC&oi=fnd&pg=PA13&dq=info:otJvizwCRqsJ:scholar.google.com&ots=ubH-ZHpGno&sig=CGpqtfPgODzPRVIrwx2YkdG92yU&redir_esc=y#v=onepage&q&f=false}
}
|
| Rivera JP and Santana R (1999), "Improving the discovery component of classifier systems by the application of estimation of distribution algorithms", In Proceedings of the students sessions, ACAI'99. Chania, Greece , pp. 43-44. |
| Abstract: This paper reports preliminary results of a Classifier Systems that uses an EDA as its discovery component. We have introduced a new method to estimate the prediction of descendant, and explained measures that help to a deeply understanding of the XCS performance. Further work includes the analysis of other classifier systems for more complex environments that would need discovery components able to exploit in a greater measure particular characteristics of the environment. |
BibTeX:
@inproceedings{Rivera_and_Santana:1999,
author = {J. P. Rivera and R. Santana},
title = {Improving the discovery component of classifier systems by the application of estimation of distribution algorithms},
booktitle = {Proceedings of the students sessions, ACAI'99},
year = {1999},
pages = {43-44},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/1999/Rivera1999a.pdf}
}
|
| Rivera JP and Santana R (2000), "Design of an algorithm based on the estimation of distributions to generate new rules in the XCS classifier system". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba, June, 2000. (ICIMAF 2000-100, CEMAFIT 2000-78) |
| Abstract: In classifier systems the genetic algorithms (GAs) have been usually employed as the discovery component. The theory of evolutionary algorithms has achieved important results nowadays, but classifier systems do not seem to be employing these advances in their own benefit. The aim of this paper is to analyze the effect of replacing the traditional discovery component of the XCS classifier system by another kind of population based search method, an Estimation Distribution Algorithm (EDA). The algorithm, which we have called CS-EDA required the implementation of a mutation-like effect with a selected mutation rate. To achieve a proper performance of XCS a new rule deletion method was developed. A more elaborated technique for the calculation of the predictions of the offspring was devised. Finally, to obtain a categorical comparison between both evolutionary algorithms it was necessary to define performance measures that permitted us to verify that the proposed algorithm performed better than the GA for the examples considered. |
BibTeX:
@techreport{Rivera_and_Santana:2000,
author = {J. P. Rivera and R. Santana},
title = {Design of an algorithm based on the estimation of distributions to generate new rules in the XCS classifier system},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {2000},
number = {ICIMAF 2000-100, CEMAFIT 2000-78}
}
|
| Castro-Jr. OR, Santana R and Pozo A (2016), "C-Multi: A competent multi-swarm approach for many-objective problems", Neurocomputing. Vol. 180, pp. 68-78. |
| Abstract: One of the major research topics in the evolutionary multi-objective community is handling a large number of objectives also known as many-objective optimization problems (MaOPs). Most existing methodologies have demonstrated success for problems with two and three objectives but face significant challenges in many-objective optimization. To tackle these challenges, a hybrid multi-swarm algorithm called C-Multi was proposed in a previous work. The project of C-Multi is based on two phases; the first uses a unique particle swarm optimization (PSO) algorithm to discover different regions of the Pareto front. The second phase uses multiple swarms to specialize on a dedicate part. On each sub-swarm, an estimation of distribution algorithm (EDA) is used to focus on convergence to its allocated region. In this study, the influence of two critical components of C-Multi, the archiving method and the number of swarms, is investigated by empirical analysis. As a result of this investigation, an improved variant of C-Multi is obtained, and its performance is compared to I-Multi, a multi-swarm algorithm that has a similar approach but does not use EDAs. Empirical results fully demonstrate the superiority of our proposed method on almost all considered test instances. |
BibTeX:
@article{Rodrigues_et_al:2016,
author = {Olacir Rodrigues-Castro-Jr. and Roberto Santana and Aurora Pozo},
title = {C-Multi: A competent multi-swarm approach for many-objective problems},
journal = {Neurocomputing},
year = {2016},
volume = {180},
pages = {68--78},
url = {https://www.sciencedirect.com/science/article/pii/S0925231215016215}
}
|
| Castro OR, Pozo A, Lozano JA and Santana R (2016), "Transfer weight functions for injecting problem information in the multi-objective CMA-ES", Memetic Computing. , pp. 1-28. Springer. |
| Abstract: The covariance matrix adaptation evolution strategy (CMA-ES) is one of the state-of-the-art evolutionary algorithms for optimization problems with continuous representation. It has been extensively applied to single-objective optimization problems, and different variants of CMA-ES have also been proposed for multi-objective optimization problems (MOPs). When applied to MOPs, the traditional steps of CMA-ES have to be modified to accommodate for multiple objectives. This fact is particularly evident when the number of objectives is higher than 3 and, with a high probability, all the solutions produced become non-dominated. An open question is to what extent information about the objective values of the non-dominated solutions can be injected in the CMA-ES model for a more effective search. In this paper, we investigate this general question using several metrics that describe the quality of the solutions already evaluated, different transfer weight functions, and a set of difficult benchmark instances including many-objective problems. We introduce a number of new strategies that modify how the probabilistic model is learned in CMA-ES. By conducting an exhaustive empirical analysis on two difficult benchmarks of many-objective functions we show that the proposed strategies to infuse information about the quality indicators into the learned models can achieve consistent improvements in the quality of the Pareto fronts obtained and enhance the convergence rate of the algorithm. Moreover, we conducted a comparison with a state-of-the-art algorithm from the literature, and achieved competitive results in problems with irregular Pareto fronts. |
BibTeX:
@article{Rodrigues_et_al:2016a,
author = {Castro, Olacir R and Pozo, Aurora and Lozano, Jose A and Santana, Roberto},
title = {Transfer weight functions for injecting problem information in the multi-objective CMA-ES},
journal = {Memetic Computing},
publisher = {Springer},
year = {2016},
pages = {1--28},
url = {https://link.springer.com/article/10.1007/s12293-016-0202-5}
}
|
| Castro OR, Lozano JA, Santana R and Pozo A (2017), "Combining CMA-ES and MOEA/DD for many-objective optimization", In Proceedings of the IEEE Congress on Evolutionary Computation, 2017. (CEC 2017). , pp. 1451-1458. |
| Abstract: Multi-objective Estimation of Distribution Algorithms (MOEDAS) have been successfully applied to solve Multi-objective Optimization Problems (MOPs) since they are able to model dependencies between variables of the problem and then sample new solutions to guide the search to promising areas. A state-of-the-art optimizer for single-objective continuous functions that also uses probabilistic modeling is the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). Different variants of CMA-ES have been proposed for MOPs however most of them are based on Pareto dominance as the main selection criterion. Recently, a new multi-objective CMA-ES called MOEA/D-CMA was proposed combining the strengths of CMA-ES with those of the multi-objective evolutionary algorithm based on decomposition (MOEA/D). Nowadays, however, researchers on MOEAs agree that combining Pareto and decomposition can be beneficial for the search on MOPs. As a result, a new MOEA has been proposed, called MOEA/DD. This algorithm modifies the MOEA/D by including a new Pareto dominance update mechanism that brings more diversity into the search. In this study, we extend the MOEA/D-CMA by replacing its update mechanism by the one of MOEA/DD. The hypothesis is that this update mechanism will improve the performance of MOEA/D-CMA as it improved MOEA/D. MOEA/D-CMA and MOEA/DD-CMA are implemented and evaluated through an experimental study. The experimental study involves two well-known families of benchmark problems whose objective numbers scale from two to fifteen. Then, an extensive statistical analysis of the results is made to extract sound, statistically supported conclusions about the performance of the algorithms as the number of objectives scales. |
BibTeX:
@inproceedings{Rodrigues_et_al:2017,
author = {Castro, Olacir R and Lozano, Jose A and Santana, Roberto and Pozo, Aurora},
title = {Combining CMA-ES and MOEA/DD for many-objective optimization},
booktitle = {Proceedings of the IEEE Congress on Evolutionary Computation, 2017. (CEC 2017)},
year = {2017},
pages = {1451--1458},
url = {https://ieeexplore.ieee.org/document/7969474}
}
|
| Castro OR, Pozo A, Lozano JA and Santana R (2017), "An investigation of clustering strategies in many-objective optimization: the I-Multi algorithm as a case study", Swarm Intelligence. Vol. 11(2), pp. 101-130. Springer. |
| Abstract: A variety of general strategies have been applied to enhance the performance of multi-objective optimization algorithms for many-objective optimization problems (those with more than three objectives). One of these strategies is to split the solutions to cover different regions of the search space (clusters) and apply an optimizer to each region with the aim of producing more diverse solutions and achieving a better distributed approximation of the Pareto front. However, the effectiveness of clustering in this context depends on a number of issues, including the characteristics of the objective functions. In this paper we show how the choice of the clustering strategy can greatly influence the behavior of an optimizer. We investigate the relation between the characteristics of a multi-objective optimization problem and the efficiency of the use of a clustering combination (clustering space, metric) in the resolution of this problem. Using as a case study the Iterated Multi-swarm (I-Multi) algorithm, a recently introduced multi-objective particle swarm optimization algorithm, we scrutinize the impact that clustering in different spaces (of variables, objectives and a combination of both) can have on the approximations of the Pareto front. Furthermore, employing two difficult multi-objective benchmarks of problems with up to 20 objectives, we evaluate the effect of using different metrics for determining the similarity between the solutions during the clustering process. Our results confirm the important effect of the clustering strategy on the behavior of multi-objective optimizers. Moreover, we present evidence that some problem characteristics can be used to select the most effective clustering strategy, significantly improving the quality of the Pareto front approximations produced by I-Multi. |
BibTeX:
@article{Rodrigues_et_al:2017a,
author = {Castro, Olacir R and Pozo, Aurora and Lozano, Jose A and Santana, Roberto},
title = {An investigation of clustering strategies in many-objective optimization: the I-Multi algorithm as a case study},
journal = {Swarm Intelligence},
publisher = {Springer},
year = {2017},
volume = {11},
number = {2},
pages = {101--130},
url = {https://link.springer.com/article/10.1007/s11721-017-0134-9}
}
|
| Roman I, Mendiburu A, Santana R and Lozano JA (2014), "Dynamic Kernel Selection Criteria for Bayesian Optimization", In 2014 NIPS Workshop on Bayesian Optimization. , pp. 1-8. |
| Abstract: In Bayesian Optimization, when using a Gaussian Process prior, some kernels adapt better than others to the objective function. This research evaluates the possibility of dynamically changing the kernel function based on the probability of improvement. Five kernel selection strategies are proposed and tested in well known synthetic functions. According to our preliminary experiments, these methods can improve the efficiency of the search when the best kernel for the problem is unknown. |
BibTeX:
@inproceedings{Roman_et_al:2014,
author = {Roman, Ibai and Mendiburu, Alexander and Santana, Roberto and Lozano, Jose A},
title = {Dynamic Kernel Selection Criteria for Bayesian Optimization},
booktitle = {2014 NIPS Workshop on Bayesian Optimization},
year = {2014},
pages = {1--8},
url = {https://bayesopt.github.io/papers/2014/paper13.pdf}
}
|
| Roman I, Santana R, Mendiburu A and Lozano JA (2015), "Kernel hautapen dinamikoa Optimizazio Bayesiarrean", In I. Ikergazte: Nazioarteko ikerketa euskaraz. Kongresuko artikulu-bilduma. , pp. 690-695. |
| Abstract: Optimizazio Bayesiarra Prozesu Gaussiarren bitartez egiten denean, kernel batzuk beste batzuk baino hobeto egokitzen dira helburu-funtziora. Lan honetan, kernel hauek dinamikoki aldatzeko aukera aztertu dugu, hobekuntza-probabilitatean oinarriturik. Kernelen hautaketa aurrera eramateko bost irizpide aurkeztu eta helburu-funtzio ezagunen bidez ebaluatu ditugu. Lortutako emaitzen arabera, irizpide hauek algoritmoaren errendimendua hobetzen dute kernel egokiena aurretiaz ezezaguna denean. |
BibTeX:
@inproceedings{Roman_et_al:2015,
author = {Roman, Ibai and Santana, Roberto and Mendiburu, Alexander and Lozano, Jose Antonio},
title = {Kernel hautapen dinamikoa Optimizazio Bayesiarrean},
booktitle = {I. Ikergazte: Nazioarteko ikerketa euskaraz. Kongresuko artikulu-bilduma},
year = {2015},
pages = {690--695},
url = {http://www.ueu.eus/download/liburua/IKERGAZTE_0.2015.pdf}
}
|
| Roman I, Santana R, Mendiburu A and Lozano JA (2019), "Sentiment analysis with genetically evolved Gaussian kernels", In Proceedings of the 2019 on Genetic and Evolutionary Computation Conference. Prague, Czech Republic , pp. 1328-1336. ACM. |
| Abstract: Sentiment analysis consists of evaluating opinions or statements based on text analysis. Among the methods used to estimate the degree to which a text expresses a certain sentiment are those based on Gaussian Processes. However, traditional Gaussian Processes methods use a predefined kernels with hyperparameters that can be tuned but whose structure can not be adapted. In this paper, we propose the application of Genetic Programming for the evolution of Gaussian Process kernels that are more precise for sentiment analysis. We use use a very flexible representation of kernels combined with a multi-objective approach that considers simultaneously two quality metrics and the computational time required to evaluate those kernels. Our results show that the algorithm can outperform Gaussian Processes with traditional kernels for some of the sentiment analysis tasks considered. |
BibTeX:
@inproceedings{Roman_et_al:2019,
author = {Ibai Roman and Roberto Santana and Alexander Mendiburu and Jose Antonio Lozano},
title = {Sentiment analysis with genetically evolved Gaussian kernels},
booktitle = {Proceedings of the 2019 on Genetic and Evolutionary Computation Conference},
publisher = {ACM},
year = {2019},
pages = {1328--1336},
url = {https://dl.acm.org/doi/10.1145/3321707.3321779}
}
|
| Roman I, Santana R, Mendiburu A and Lozano JA (2019), "Evolving Gaussian Process kernels for translation editing effort estimation", In Proceedings of the Learning and Intelligent Optimization Conference (LION). Chania, Greece , pp. 304-318. ACM. |
| Abstract: In many Natural Language Processing problems the combination of machine learning and optimization techniques is essential. One of these problems is estimating the effort required to improve, under direct human supervision, a text that has been translated using a machine translation method. Recent developments in this area have shown that Gaussian Processes can be accurate for post-editing effort prediction. However, the Gaussian Process kernel has to be chosen in advance, and this choice influences the quality of the prediction. In this paper, we propose a Genetic Programming algorithm to evolve kernels for Gaussian Processes. We show that the combination of evolutionary optimization and Gaussian Processes removes the need for a-priori specification of the kernel choice, and achieves predictions that, in many cases, outperform those obtained with fixed kernels. |
BibTeX:
@inproceedings{Roman_et_al:2019a,
author = {Ibai Roman and Roberto Santana and Alexander Mendiburu and Jose Antonio Lozano},
title = {Evolving Gaussian Process kernels for translation editing effort estimation},
booktitle = {Proceedings of the Learning and Intelligent Optimization Conference (LION)},
publisher = {ACM},
year = {2019},
pages = {304--318},
url = {https://link.springer.com/chapter/10.1007/978-3-030-38629-0_25}
}
|
| Roman I, Mendiburu A, Santana R and Lozano JA (2019), "Bayesian Optimization Approaches for Massively Multi-modal Problems", In International Conference on Learning and Intelligent Optimization (LION). , pp. 383-397. |
| Abstract: The optimization of massively multi-modal functions is a challenging task, particularly for problems where the search space can lead the optimization process to local optima. While evolutionary algorithms have been extensively investigated for these optimization problems, Bayesian Optimization algorithms have not been explored to the same extent. In this paper, we study the behavior of Bayesian Optimization as part of a hybrid approach for solving several massively multi-modal functions. We use well-known benchmarks and metrics to evaluate how different variants of Bayesian Optimization deal with multi-modality. |
BibTeX:
@inproceedings{Roman_et_al:2019b,
author = {Roman, Ibai and Mendiburu, Alexander and Santana, Roberto and Lozano, Jose A},
title = {Bayesian Optimization Approaches for Massively Multi-modal Problems},
booktitle = {International Conference on Learning and Intelligent Optimization (LION)},
year = {2019},
pages = {383--397},
url = {https://link.springer.com/chapter/10.1007/978-3-030-38629-0_31}
}
|
| Roman I, Santana R, Mendiburu A and Lozano JA (2019), "An Experimental Study in Adaptive Kernel Selection for Bayesian Optimization", IEEE Access. Vol. 7(8936460), pp. 184394-184302. IEEE Press. |
| Abstract: Bayesian Optimization has been widely used along with Gaussian Processes for solving expensive-to-evaluate black-box optimization problems. Overall, this approach has shown good results, and particularly for parameter tuning of machine learning algorithms. Nonetheless, Bayesian Optimization has to be also configured to achieve the best possible performance, being the selection of the kernel function a crucial choice. This paper investigates the convenience of adaptively changing the kernel function during the optimization process, instead of fixing it a priori. Six adaptive kernel selection strategies are introduced and tested in well-known synthetic and real-world optimization problems. In order to provide a more complete evaluation of the proposed kernel selection variants, two major kernel parameter setting approaches have been tested. According to our results, apart from having the advantage of removing the selection of the kernel out of the equation, adaptive kernel selection criteria show a better performance than fixed-kernel approaches. |
BibTeX:
@article{Roman_et_al:2019c,
author = {Roman, Ibai and Santana, Roberto and Mendiburu, Alexander and Lozano, Jose A},
title = {An Experimental Study in Adaptive Kernel Selection for Bayesian Optimization},
journal = {IEEE Access},
publisher = {IEEE Press},
year = {2019},
volume = {7},
number = {8936460},
pages = {184394--184302},
url = {https://ieeexplore.ieee.org/document/8936460}
}
|
| Roman I, Santana R, Mendiburu A and Lozano JA (2020), "In-depth analysis of SVM kernel learning and its components", Neural Computing and Applications. , pp. 1-20. Springer. |
| Abstract: The performance of support vector machines in nonlinearly separable classification problems strongly relies on the kernel function. Toward an automatic machine learning approach for this technique, many research outputs have been produced dealing with the challenge of automatic learning of good-performing kernels for support vector machines. However, these works have been carried out without a thorough analysis of the set of components that influence the behavior of support vector machines and their interaction with the kernel. These components are related in an intricate way and it is difficult to provide a comprehensible analysis of their joint effect. In this paper, we try to fill this gap introducing the necessary steps in order to understand these interactions and provide clues for the research community to know where to place the emphasis. First of all, we identify all the factors that affect the final performance of support vector machines in relation to the elicitation of kernels. Next, we analyze the factors independently or in pairs and study the influence each component has on the final classification performance, providing recommendations and insights into the kernel setting for support vector machines. |
BibTeX:
@article{Roman_et_al:2020,
author = {Roman, Ibai and Santana, Roberto and Mendiburu, Alexander and Lozano, Jose A},
title = {In-depth analysis of SVM kernel learning and its components},
journal = {Neural Computing and Applications},
publisher = {Springer},
year = {2020},
pages = {1--20},
url = {https://link.springer.com/article/10.1007/s00521-020-05419-z}
}
|
| Roman I, Santana R, Mendiburu A and Lozano JA (2021), "Evolution of Gaussian Process kernels for machine translation post-editing effort estimation", Annals of Mathematics and Artificial Intelligence. Vol. 89, pp. 835-856. Springer. |
| Abstract: In many Natural Language Processing problems the combination of machine learning and optimization techniques is essential. One of these problems is the estimation of the human effort needed to improve a text that has been translated using a machine translation method. Recent advances in this area have shown that Gaussian Processes can be effective in post-editing effort prediction. However, Gaussian Processes require a kernel function to be defined, the choice of which highly influences the quality of the prediction. On the other hand, the extraction of features from the text can be very labor-intensive, although recent advances in sentence embedding have shown that this process can be automated. In this paper, we use a Genetic Programming algorithm to evolve kernels for Gaussian Processes to predict post-editing effort based on sentence embeddings. We show that the combination of evolutionary optimization and Gaussian Processes removes the need for a-priori specification of the kernel choice, and, by using a multi-objective variant of the Genetic Programming approach, kernels that are suitable for predicting several metrics can be learned. We also investigate the effect that the choice of the sentence embedding method has on the kernel learning process. |
BibTeX:
@article{Roman_et_al:2021,
author = {Roman, Ibai and Santana, Roberto and Mendiburu, Alexander and Lozano, Jose A},
title = {Evolution of Gaussian Process kernels for machine translation post-editing effort estimation},
journal = {Annals of Mathematics and Artificial Intelligence},
publisher = {Springer},
year = {2021},
volume = {89},
pages = {835-856},
url = {https://link.springer.com/article/10.1007/s10472-021-09751-5}
}
|
| Santana R and Alba E (2001), "Generating test matrices to evaluate the performance of strategies to search typical testors". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba, January, 2001. (ICIMAF 2000-130) |
| Abstract: Testors and particularly typical testors, have been used in feature selection and supervised classification problems. Deterministic algorithms have usually been used to find typical testors. Recently, a new approach based on evolutionary algorithms has been developed. A common problem to test the behavior of both approaches is the necessity of knowing, in advance, the number of typical testors of a given basic matrix. For an arbitrary matrix, this number can not be known unless all typical testors have been found. Therefore, this paper introduces, for the first time, a strategy to generate basic matrices for which the number of typical testors is known without to find them. This method is illustrated with some examples. |
BibTeX:
@techreport{Santana_and_Alba:2001a,
author = {Roberto Santana and Eduardo Alba},
title = {Generating test matrices to evaluate the performance of strategies to search typical testors},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {2001},
number = {ICIMAF 2000-130},
url = {https://revistas.usfq.edu.ec/index.php/avances/article/view/23}
}
|
| Santana R and Lozano JA (2017), "Different scenarios for survival analysis of evolutionary algorithms", In Proceedings of the Genetic and Evolutionary Computation Conference. , pp. 825-832. |
| Abstract: Empirical analysis of evolutionary algorithms (EAs) behavior is usually approached by computing relatively simple descriptive statistics like mean fitness and mean number of evaluations to convergence, or more theoretically sound statistical tests for finding significant differences between algorithms. However, these analyses do not consider situations where the EA failed to finish due to numerical errors or excessive computational time. Furthermore, the ability of an EA to continuously make search improvements is usually overlooked. In this paper we propose the use of the theory from survival analysis for empirically investigating the behavior of EAs, even in situations where not all the experiments finish in a reasonable time. We introduce two scenarios for the application of survival analysis in EAs. Survival trees, a machine learning technique adapted to the survival analysis scenario, are applied to automatically identify combinations of EA parameters with similar effect in the behavior of the algorithm. |
BibTeX:
@inproceedings{Santana_and_Lozano:2017,
author = {Santana, Roberto and Lozano, Jose A},
title = {Different scenarios for survival analysis of evolutionary algorithms},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference},
year = {2017},
pages = {825--832},
url = {https://dl.acm.org/doi/10.1145/3071178.3071250}
}
|
| Santana R and Mendiburu A (2013), "Model-based template-recombination in Markov network estimation of distribution algorithms for problems with discrete representation", In 2013 Third World Congress on Information and Communication Technologies (WICT 2013). , pp. 170-175. |
| Abstract: While estimation of distribution algorithms (EDAs) based on Markov networks usually incorporate efficient methods to learn undirected probabilistic graphical models (PGMs) from data, the methods they use for sampling the PGMs are computationally costly. In addition, methods for generating solutions in Markov network based EDA frequently discard information contained in the model to gain in efficiency. In this paper we propose a new method for generating solutions that uses the Markov network structure as a template for crossover. The new algorithm is evaluated on discrete deceptive functions of various degrees of difficulty and Ising instances. |
BibTeX:
@inproceedings{Santana_and_Mendiburu:2013,
author = {Santana, Roberto and Mendiburu, Alexander},
title = {Model-based template-recombination in Markov network estimation of distribution algorithms for problems with discrete representation},
booktitle = {2013 Third World Congress on Information and Communication Technologies (WICT 2013)},
year = {2013},
pages = {170--175},
url = {https://ieeexplore.ieee.org/document/7113130?signout=success}
}
|
| Santana R and Mühlenbein H (2002), "Blocked Stochastic Sampling versus Estimation of Distribution Algorithms", In Proceedings of the 2002 Congress on Evolutionary Computation CEC-2002. Vol. 2, pp. 1390-1395. IEEE press. |
| Abstract: The Boltzmann distribution is a good candidate for a search distribution for optimization problems. We compare two methods to approximate the Boltzmann distribution-Estimation of Distribution Algorithms (EDA) and Markov Chain Monte Carlo methods (MCMC). It turns out that in the space of binary functions even blocked MCMC methodsoutperform EDA on a small class of problems only. In these cases a temperature of T=0 performed best |
BibTeX:
@inproceedings{Santana_and_Muehlenbein:2002,
author = {R. Santana and H. Mühlenbein},
title = {Blocked Stochastic Sampling versus Estimation of Distribution Algorithms},
booktitle = {Proceedings of the 2002 Congress on Evolutionary Computation CEC-2002},
publisher = {IEEE press},
year = {2002},
volume = {2},
pages = {1390-1395},
url = {https://ieeexplore.ieee.org/document/1004446}
}
|
| Santana R and Ochoa A (1999), "A Constraint Univariate Marginal Distribution Algorithm". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba (ICIMAF 99-76, CENIA 99-04) |
| Abstract: This paper proposes a new optimization algorithm to deal with binary constraint problems. The algorithm is based in the Univariate Marginal Distribution Algorithm. We apply our approach to the optimization of functions with different characteristics. For the test functions considered we show the superiority of our algorithm to traditional population search methods that have been used to solve these kind of problems. We report some particular features exhibited by the algorithm and discuss extensions that could make of it a still more powerful optimization tool. |
BibTeX:
@techreport{Santana_and_Ochoa:1999,
author = {R. Santana and A. Ochoa},
title = {A Constraint Univariate Marginal Distribution Algorithm},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {1999},
number = {ICIMAF 99-76, CENIA 99-04},
url = {https://www.researchgate.net/publication/333718972_A_Constraint_Univariate_Marginal_Distribution_Algorithm}
}
|
| Santana R and Ochoa A (1999), "On Estimation Distribution Algorithms", In Proceedings of the Genetic and Evolutionary Computation Conference GECCO-1999, Workshop Program. Orlando, FL , pp. 402. Morgan Kaufmann Publishers, San Francisco, CA. |
| Abstract: The main goal of the dissertation research is the design of efficient Estimation Distribution Algorithms based on non simply connected probabilistic networks. This work is part of a more ambitious ongoing research on Low Cost Evolutionary Algorithms (LCEA). Other important areas to be covered with the dissertation are: 1) The integration of different theories about GA in order to explain successful results of EDA in the context of evolutionary computation. 2) Define ways of incorporating knowledge about the problem domain, prior to the beginning and during the function optimization . 3) To study the recognized sources of hardness for GA’s in the framework of EDA. 4) Define measures to evaluate the quality of the different search strategies used. We now describe some aspect of the current research in these topics. |
BibTeX:
@inproceedings{Santana_and_Ochoa:1999a,
author = {R. Santana and A. Ochoa},
editor = {Wu A. S.},
title = {On Estimation Distribution Algorithms},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference GECCO-1999, Workshop Program},
publisher = {Morgan Kaufmann Publishers, San Francisco, CA},
year = {1999},
pages = {402}
}
|
| Santana R and Ochoa A (1999), "Dealing with constraints with estimation of distribution algorithms: The univariate case", In Proceedings of the Second Symposium on Artificial Intelligence (CIMAF-99). Havana, Cuba, March, 1999. , pp. 378-384. |
| Abstract: This paper proposes a new optimization algorithm to deal with binary constraint problems. The algorithm is based in the Univariate Marginal Distribution Algorithm. We apply our approach to the optimization of functions with different characteristics. For the test functions considered we show the superiority of our algorithm to traditional population search methods that have been used to solve these kind of problems. We report some particular features exhibited by the algorithm and discuss extensions that could make of it a still more powerful optimization tool. |
BibTeX:
@inproceedings{Santana_and_Ochoa:1999b,
author = {R. Santana and A. Ochoa},
editor = {A. Ochoa and M. R. Soto and R. Santana},
title = {Dealing with constraints with estimation of distribution algorithms: The univariate case},
booktitle = {Proceedings of the Second Symposium on Artificial Intelligence (CIMAF-99)},
year = {1999},
pages = {378-384},
url = {https://www.researchgate.net/publication/333718972_A_Constraint_Univariate_Marginal_Distribution_Algorithm}
}
|
| Santana R and de León EP (1997), "A hybrid genetic algorithm for a Hamiltonian path problem", In Proceedings of the First Symposium on Artificial Intelligence (CIMAF-97). Havana, Cuba, March, 1997. , pp. 126-132. Editora de la Academia de Ciencias de Cuba.
[BibTeX] |
BibTeX:
@inproceedings{Santana_and_Ponce:1997,
author = {R. Santana and E. Ponce de León},
title = {A hybrid genetic algorithm for a Hamiltonian path problem},
booktitle = {Proceedings of the First Symposium on Artificial Intelligence (CIMAF-97)},
publisher = {Editora de la Academia de Ciencias de Cuba},
year = {1997},
pages = {126-132}
}
|
| Santana R and de León EP (1998), "A conceptual model for detecting structures in graphs using evolutionary optimization algorithms". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba, December, 1998. (ICIMAF 98-69, CENIA 98-03) |
| Abstract: Search and identification of structures on graphs have been topics extensively considered in the literature. Different approaches are known that face these problems as functions optimization. Most of them emphasize on the search of a particular structure (e.g. cliques, spanning trees, hamiltonian paths). In the same way, functions are conceived to be optimized using just one method (e.g. gradient, annealing, genetic algorithms.) Here we analyze the convenience of defining families of functions to be used in the search of different structures. Starting from the identification of a dissection on a graph we propose a family of functions that shows good results in the search of a wide set of structures on graphs. A hierarchy that associates the function definition to a constraint addition process is introduced. We also deal with the question of an optimal representation for the solutions space. Finally we present the results obtained using two different optimization algorithms. A heuristic hill-climbing and a population based search method, which are utilized to deal with the multi-objective character of our constraint optimization problem. |
BibTeX:
@techreport{Santana_and_Ponce:1998,
author = {R. Santana and E. Ponce de León},
title = {A conceptual model for detecting structures in graphs using evolutionary optimization algorithms},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {1998},
number = {ICIMAF 98-69, CENIA 98-03},
url = {https://www.researchgate.net/publication/333719090_A_conceptual_model_for_detecting_structures_on_graphs_using_evolutionary_optimization_algorithms}
}
|
| Santana R and de León EP (1998), "An evolutionary optimization approach for detecting structures on graphs", In Smart Engineering System Design: Neural Network, Fuzzy Logic, Rough Sets and Evolutionary Programming. , pp. 371-376. ASME press. |
| Abstract: This paper introduces a function optimization approach for detecting structures on graphs. Starting from the identification of a dissection on a graph, we propose a family of functions whose parameters vary according to the different structures to be searched. We also deal with the question of an optimal representation for the solutions. Finally we present the results obtained using two different optimization algorithms. A heuristic hill-climbing and a population based search method are presented, which are utilized to deal with the multi-objective character of our constraint optimization problem. |
BibTeX:
@inproceedings{Santana_and_Ponce:1998a,
author = {R. Santana and E. Ponce de León},
editor = {Dagli and Akay and Buczac and Ersoy and Fernandez},
title = {An evolutionary optimization approach for detecting structures on graphs},
booktitle = {Smart Engineering System Design: Neural Network, Fuzzy Logic, Rough Sets and Evolutionary Programming},
publisher = {ASME press},
year = {1998},
pages = {371-376},
url = {https://www.researchgate.net/publication/262522456_An_evolutionary_optimization_approach_for_detecting_structures_on_graphs}
}
|
| Santana R and de León EP (1998), "Defining families of functions to be used in the search of different structures on graphs", In Proceedings of the IX Congreso Iberoamericano de Investigación Operativa. |
| Abstract: Search and identification of structures on graphs have been topics extensively considered in the literature. Different approaches are known that face these problems as functions optimization. Most of them emphasize on the search of a particular structure (e.g. cliques, spanning trees, Hamiltonian paths). In the same way, functions are conceived to be optimized using just one method ( e.g. gradient, annealing, genetic algorithms, etc.) Here we introduce families of functions to be used in the search of different structures with several optimization algorithms. Starting from the identification of a dissection on a graph we propose a family of functions that shows good results in the search of a wide set of structures on graphs. We also consider the question of an optimal representation for the solutions space. We have found structures like triangulations, perfect matching and Hamiltonian circuits using different optimization methods. The behavior of the used algorithms is analyzed. |
BibTeX:
@inproceedings{Santana_and_Ponce:1998b,
author = {R. Santana and E. Ponce de León},
title = {Defining families of functions to be used in the search of different structures on graphs},
booktitle = {Proceedings of the IX Congreso Iberoamericano de Investigación Operativa},
year = {1998},
url = {https://www.researchgate.net/publication/333719090_A_conceptual_model_for_detecting_structures_on_graphs_using_evolutionary_optimization_algorithms}
}
|
| Santana R and Shakya S (2012), "Probabilistic Graphical Models and Markov Networks", In Markov Networks in Evolutionary Computation. , pp. 3-19. Springer. |
| Abstract: This chapter introduces probabilistic graphical models and explain their use for modelling probabilistic relationships between variables in the context of optimisation with EDAs. We focus on Markov networks models and review different algorithms used to learn and sample Markov networks. Other probabilistic graphical models are also reviewed and their differences with Markov networks are analysed. |
BibTeX:
@incollection{Santana_and_Shakya:2012a,
author = {R. Santana and S. Shakya},
editor = {S. Shakya and R. Santana},
title = {Probabilistic Graphical Models and Markov Networks},
booktitle = {Markov Networks in Evolutionary Computation},
publisher = {Springer},
year = {2012},
pages = {3-19},
url = {http://dx.doi.org/10.1007/978-3-642-28900-2_1}
}
|
| Santana R and Shakya S (2020), "Dynamic programming operators for bi-objective TTP problem", In 2020 IEEE Congress on Evolutionary Computation (CEC). Glasgow, UK , pp. 1-8. |
| Abstract: The traveling thief problem (TTP) has emerged as a realistic multi-component problem that poses a number of challenges to traditional optimizers. In this paper we propose different ways to incorporate dynamic programming (DP) as a local optimization operator of population-based approaches to the biobjective TTP. The DP operators use different characterizations of the TTP instance to search for packing plans that improve the best current solutions. We evaluate the efficiency of the DP-based operators using TTP instances of up to 33810 cities and 338100 items, and compare the results of the DP operators with state-of-the-art algorithms for these instances. Our results show that DP-based approaches, applied individually and in combination with other types of operators, can produce good approximations of the Pareto sets for these problems. |
BibTeX:
@inproceedings{Santana_and_Shakya:2020,
author = {R. Santana and S. Shakya},
title = {Dynamic programming operators for bi-objective TTP problem},
booktitle = {2020 IEEE Congress on Evolutionary Computation (CEC)},
year = {2020},
pages = {1-8},
url = {https://ieeexplore.ieee.org/abstract/document/9185829}
}
|
| Santana R, Mendiburu A and Lozano JA (2014), "Customized Selection in Estimation of Distribution Algorithms", In Proceedings of the 10th International Conference Simulated Evolution and Learning (SEAL-2014). , pp. 94-105. Springer. |
| Abstract: Selection plays an important role in estimation of distribution algorithms. It determines the solutions that will be modeled to represent the promising areas of the search space. There is a strong relationship between the strength of selection and the type and number of dependencies that are captured by the models. In this paper we propose to use different selection probabilities to learn the structural and parametric components of the probabilistic graphical models. Customized selection is introduced as a way to enhance the effect of model learning in the exploratory and exploitative aspects of the search. We use a benchmark of over 15,000 instances of a simplified protein model to illustrate the gains in using customized selection. |
BibTeX:
@incollection{Santana_e_al:2014,
author = {Santana, Roberto and Mendiburu, Alexander and Lozano, Jose A},
title = {Customized Selection in Estimation of Distribution Algorithms},
booktitle = {Proceedings of the 10th International Conference Simulated Evolution and Learning (SEAL-2014)},
publisher = {Springer},
year = {2014},
pages = {94--105},
url = {https://link.springer.com/chapter/10.1007/978-3-319-13563-2_9}
}
|
| Santana R, McDonald RB and Katzgraber HG (2014), "A probabilistic evolutionary optimization approach to compute quasiparticle braids", In Proceedings of the 10th International Conference Simulated Evolution and Learning (SEAL-2014). , pp. 13-24. Springer. |
BibTeX:
@incollection{Santana_et_al::2014a,
author = {R. Santana and R. B. McDonald and H. G. Katzgraber},
title = {A probabilistic evolutionary optimization approach to compute quasiparticle braids},
booktitle = {Proceedings of the 10th International Conference Simulated Evolution and Learning (SEAL-2014)},
publisher = {Springer},
year = {2014},
pages = {13--24},
url = {https://arxiv.org/abs/1410.0602}
}
|
| Santana R, de León EP and Ochoa A (1999), "The incident edge model", In Proceedings of the Second Symposium on Artificial Intelligence (CIMAF-99). Havana, Cuba, March, 1999. , pp. 352-359. |
| Abstract: In this paper we introduce the incident edge model, a fitness function model for a large set of problems defined on graphs. The model can be used to define functions whose optimization led to the finding of different structures on graphs. The model can also be useful to determine the best optimization algorithm for a given problem. As an example of the application of our model we describe an optimization approach for the problem of finding the dissection of a graph. For the optimization of this additively decomposable function we use the Factorized Distribution Algorithm. We focus on the way that different factorizations of the probability distribution can influence the behavior of the Factorized Distribution Algorithm for the dissection problem. |
BibTeX:
@inproceedings{Santana_et_al:1999,
author = {R. Santana and E. Ponce de León and A. Ochoa},
editor = {A. Ochoa and M. R. Soto and R. Santana},
title = {The incident edge model},
booktitle = {Proceedings of the Second Symposium on Artificial Intelligence (CIMAF-99)},
year = {1999},
pages = {352-359},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/1999/cimaf5.pdf}
}
|
| Santana R, Ochoa A and Soto MR (1999), "Evolutionary Algorithms for Dynamic Optimization Problems: An approach using Evolutionary Theory and the Incident Edge Model", In Proceedings of the Genetic and Evolutionary Computation Conference GECCO-1999, Workshop Program. Orlando, FL , pp. 149-152. Morgan Kaufmann Publishers, San Francisco, CA. |
| Abstract: In this paper we analyze the use of Evolutionary Algorithms (EAs) for Dynamic Optimization Problems (DOP). We show how research on Evolutionary Theory can throw light to the question of characterizing dynamic problems. We present arguments in favor of using fitness function models as benchmark for the study of DOP. As an example we present the Incident Edge Model, and discuss how different kinds of dynamics for problems defined on graphs can be translated to the model. Finally we apply an EA, the Constraint Univariate Marginal Distribution Algorithm, for the problem of finding the spanning trees of graphs that change through time. We show that efficient EAs should be able to employ information about changes in the environment to guide the search. |
BibTeX:
@inproceedings{Santana_et_al:1999a,
author = {R. Santana and A. Ochoa and M. R. Soto},
editor = {A. S. Wu},
title = {Evolutionary Algorithms for Dynamic Optimization Problems: An approach using Evolutionary Theory and the Incident Edge Model},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference GECCO-1999, Workshop Program},
publisher = {Morgan Kaufmann Publishers, San Francisco, CA},
year = {1999},
pages = {149--152}
}
|
| Santana R, Pereira FB, Costa E, Ochoa A, Machado P, Cardoso A and Soto MR (2000), "Probabilistic Evolution and the busy beaver Problem", In Late Breaking Papers at the 2000 Genetic and Evolutionary Computation Conference GECCO-2000. Las Vegas, Nevada, USA, 8 July, 2000. , pp. 261-268. |
| Abstract: We discuss the use of probabilistic evolution in an important class of problems based on Turing Machines, namely the famous Busy Beaver. Despite the bad properties of this problem for a probabilistic solution: nonbinary representation and variable associations with a strongly connected graph-like structure, our algorithm seems to outperform previous evolutionary computation approaches. |
BibTeX:
@inproceedings{Santana_et_al:2000,
author = {Roberto Santana and Francisco B. Pereira and Ernesto Costa and Alberto Ochoa and Penousal Machado and Amilcar Cardoso and Marta Rosa Soto},
editor = {Darrell Whitley},
title = {Probabilistic Evolution and the busy beaver Problem},
booktitle = {Late Breaking Papers at the 2000 Genetic and Evolutionary Computation Conference GECCO-2000},
year = {2000},
pages = {261--268},
url = {https://cdv.dei.uc.pt/wp-content/uploads/2014/03/sors+00.pdf}
}
|
| Santana R, Ochoa A and Soto MR (2000), "A Factorized Distribution Algorithm of bounded complexity for integer problems". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba, December, 2000. (ICIMAF 2000-, CENIA 2000-) |
| Abstract: A Factorized Distribution algorithm that use up to pairwise dependencies for the optimization of integer problems is introduced. Our proposal combines classical methods for structural learning of dependencies with a procedure that approximates the bivariate marginals by sampling the data using auxiliary tables. The algorithm overperforms the Univariate Marginal Distribution Algorithm for the integer problems tested. |
BibTeX:
@techreport{Santana_et_al:2000a,
author = {R. Santana and A. Ochoa and M. R. Soto},
title = {A Factorized Distribution Algorithm of bounded complexity for integer problems},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {2000},
number = {ICIMAF 2000-, CENIA 2000-},
url = {https://www.researchgate.net/profile/Roberto-Santana/publication/269105263_A_Factorized_Distribution_Algorithm_of_bounded_complexity_for_integer_problems/links/5d00b83092851c874c5fcee8/A-Factorized-Distribution-Algorithm-of-bounded-complexity-for-integer-problems.pdf}
}
|
| Santana R, Pereira FB, Costa E, Ochoa A, Machado P, Cardoso A and Soto MR (2000), "Probabilistic Evolution and the Busy Beaver Problem", In Proceedings of the Genetic and Evolutionary Computation Conference GECCO-2000. Las Vegas, Nevada, USA, July, 2000. , pp. 380. |
| Abstract: We discuss the use of probabilistic evolution in an important class of problems based on Turing Machines, namely the famous Busy Beaver. Despite the bad properties of this problem for a probabilistic solution: nonbinary representation and variable associations with a strongly connected graph-like structure, our algorithm seems to outperform previous evolutionary computation approaches. |
BibTeX:
@inproceedings{Santana_et_al:2000b,
author = {Roberto Santana and Francisco B. Pereira and Ernesto Costa and Alberto Ochoa and Penousal Machado and Amilcar Cardoso and Marta Rosa Soto},
title = {Probabilistic Evolution and the Busy Beaver Problem},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference GECCO-2000},
year = {2000},
pages = {380},
url = {https://dl.acm.org/doi/pdf/10.5555/2933718.2933781}
}
|
| Santana R, Ochoa A and Soto MR (2001), "The mixture of trees factorized distribution algorithm". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba, January, 2001. (ICIMAF 2000-129) |
| Abstract: This paper introduces the Mixtures of Trees Factorized Distribution Algorithm (MT-FDA). It is based on a mixture of trees distribution and the Estimation Maximization learning algorithm. The probabilistic model and the learning procedure of the MT-FDA differ to previous proposals of probabilistic modeling in the context of Evolutionary Computation. Preliminary results show that the MT-FDA overperforms Factorized Distribution Algorithms that use up to second order statistics. It is also competitive, and some times superior to Bayesian Factorized Distribution Algorithms. The paper illustrates how the MT-FDA can incorporate information about particular features of the search space by conveniently selecting the mixture of trees parameters. |
BibTeX:
@techreport{Santana_et_al:2001,
author = {R. Santana and A. Ochoa and M. R. Soto},
title = {The mixture of trees factorized distribution algorithm},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {2001},
number = {ICIMAF 2000-129},
url = {https://dl.acm.org/doi/pdf/10.5555/2955239.2955322}
}
|
| Santana R, Ochoa A and Soto MR (2001), "The mixture of trees factorized distribution algorithm", In Proceedings of the Genetic and Evolutionary Computation Conference GECCO-2001. San Francisco, CA , pp. 543-550. Morgan Kaufmann Publishers. |
| Abstract: This paper introduces a Factorized Distribution Algorithm based on a mixture of trees distribution. The probabilistic model and the learning algorithm used differs to previous uses of probabilistic modeling in the context of Evolutionary Computation. Preliminary results show the algorithm is competitive, and some times superior to other Factorized Distribution Algorithms. We also illustrate how particular features of the search space can be employed during the search by conveniently selecting the mixture of trees parameters. |
BibTeX:
@inproceedings{Santana_et_al:2001b,
author = {R. Santana and A. Ochoa and M. R. Soto},
editor = {L. Spector and E. Goodman and A. Wu and W.B. Langdon and H.M. Voigt and M. Gen and S. Sen and M. Dorigo and S. Pezeshk and M. Garzon and E. Burke},
title = {The mixture of trees factorized distribution algorithm},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference GECCO-2001},
publisher = {Morgan Kaufmann Publishers},
year = {2001},
pages = {543--550},
url = {https://dl.acm.org/doi/pdf/10.5555/2955239.2955322}
}
|
| Santana R, Ochoa A and Soto MR (2001), "Factorized Distribution Algorithms for functions with unitation constraints", In Evolutionary Computation and Probabilistic Graphical Models. Proceedings of the Third Symposium on Adaptive Systems (ISAS-2001). Havana, Cuba, March, 2001. , pp. 158-165. |
| Abstract: The class of non overlapping additively decomposed functions subject to unitation constraints are of interest for studying the behavior of Factorized Distribution Algorithms (FDAs) in constrained problems. In this paper we define a theoretical framework for the analysis of Constraint FDAs (CFDAs). This framework is used to investigate the factors that could explain the behavior of FDAs for the class of functions under consideration. Empirical evidence is shown to demonstrate our assertions. |
BibTeX:
@inproceedings{Santana_et_al:2001c,
author = {R. Santana and A. Ochoa and M. R. Soto},
title = {Factorized Distribution Algorithms for functions with unitation constraints},
booktitle = {Evolutionary Computation and Probabilistic Graphical Models. Proceedings of the Third Symposium on Adaptive Systems (ISAS-2001)},
year = {2001},
pages = {158-165},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/2001/Isas_Constraints_lastnew.pdf}
}
|
| Santana R, Ochoa A and Soto MR (2001), "A Factorized Distribution Algorithm for problems with integer representation", In Proceedings of the Genetic and Evolutionary Computation Conference GECCO-2001. San Francisco, CA , pp. 780. Morgan Kaufmann Publishers. |
| Abstract: In this poster we present a Factorized Distribution Algorithm (FDA) that considers up to second order statistics, and permits to carry out the optimization of integer problems with a high cardinality of the variables. |
BibTeX:
@inproceedings{Santana_et_al:2001d,
author = {R. Santana and A. Ochoa and M. R. Soto},
editor = {L. Spector and E. Goodman and A. Wu and W.B. Langdon and H.M. Voigt and M. Gen and S. Sen and M. Dorigo and S. Pezeshk and M. Garzon and E. Burke},
title = {A Factorized Distribution Algorithm for problems with integer representation},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference GECCO-2001},
publisher = {Morgan Kaufmann Publishers},
year = {2001},
pages = {780},
url = {https://dl.acm.org/doi/pdf/10.5555/2955239.2955377}
}
|
| Santana R, Ochoa A and Soto MR (2001), "On the use of Factorized Distribution Algorithms for problems defined on graphs", In Electronic Notes in Discrete Mathematics. Vol. 8 Elsevier. |
| Abstract: This short paper surveys current work on the use of Factorized Distribution Algorithms for the solution of combinatorial optimization problems defined on graphs. We also advance a number of approaches for future work along this line. |
BibTeX:
@inproceedings{Santana_et_al:2001e,
author = {Roberto Santana and Alberto Ochoa and Marta R. Soto},
editor = {Hajo Broersma and Ulrich Faigle and Johann Hurink and Stefan Pickl},
title = {On the use of Factorized Distribution Algorithms for problems defined on graphs},
booktitle = {Electronic Notes in Discrete Mathematics},
publisher = {Elsevier},
year = {2001},
volume = {8},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/1999/FDA_Graphs.pdf}
}
|
| Santana R, Ochoa A and Soto MR (2002), "Solving problems with integer representation using a tree based factorized distribution algorithm", In Electronic Proceedings of the First International NAISO Congress on Neuro Fuzzy Technologies. NAISO Academic Press. |
| Abstract: In this paper a tree based Factorized Distribution Algorithm for the solution of integer problems is introduced. Our proposal combines classical methods for structural learning of dependencies with a a procedures that approximates the bivariate marginals by sampling the data using auxiliary tables. Experiments done for a number of problems with an integer representation show evidence of the superiority of the algorithm with respect to the Univariate Marginal Distribution Algorithm. |
BibTeX:
@inproceedings{Santana_et_al:2002,
author = {R. Santana and A. Ochoa and M. R. Soto},
title = {Solving problems with integer representation using a tree based factorized distribution algorithm},
booktitle = {Electronic Proceedings of the First International NAISO Congress on Neuro Fuzzy Technologies},
publisher = {NAISO Academic Press},
year = {2002},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/2002/Hav2002Final1.pdf}
}
|
| Santana R, Larrañaga P and Lozano JA (2004), "Protein folding in 2-dimensional lattices with estimation of distribution algorithms", In Proceedings of the First International Symposium on Biological and Medical Data Analysis. Barcelona Vol. 3337, pp. 388-398. Springer. |
| Abstract: This paper introduces a new type of evolutionary computation algorithm based on probability distributions for the solution of two simplified protein folding models. The relationship of the introduced algorithm with previous evolutionary methods used for protein folding is discussed. A number of experiments for difficult instances of the models under analysis is presented. For the instances considered, the algorithm is shown to outperform previous evolutionary optimization methods. |
BibTeX:
@inproceedings{Santana_et_al:2004,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
title = {Protein folding in 2-dimensional lattices with estimation of distribution algorithms},
booktitle = {Proceedings of the First International Symposium on Biological and Medical Data Analysis},
publisher = {Springer},
year = {2004},
volume = {3337},
pages = {388--398},
url = {http://dx.doi.org/10.1007/b104033}
}
|
| Santana R, Larrañaga P and Lozano JA (2005), "Properties of Kikuchi approximations constructed from clique based decompositions". Research Report at: Department of Computer Science and Artificial Intelligence, University of the Basque Country., April, 2005. (EHU-KZAA-IK-2/05) |
| Abstract: Kikuchi approximations constructed from clique-based decompositions can be used to calculate suitable approximations of probability distributions. They can be applied in domains such as probabilistic modeling, supervised and unsupervised classi cation, and evolutionary algorithms. This paper introduces a number of properties of these approximations. Pairwise and local Markov properties of the Kikuchi approximations are proved. We prove that, even if the global Markov property is not satisfied in the general case, it is possible to decompose the Kikuchi approximation in the product of local Kikuchi approximations defined on a decomposition of the graph. Partial Kikuchi approximations are introduced. Additionally, the paper clarifies the place of clique-based decompositions in relation to other techniques inspired by methods from statistical physics, and discusses the application of the results introduced in the paper for the conception of Kikuchi approximation learning algorithms. |
BibTeX:
@techreport{Santana_et_al:2005a,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
title = {Properties of Kikuchi approximations constructed from clique based decompositions},
school = {Department of Computer Science and Artificial Intelligence, University of the Basque Country},
year = {2005},
number = {EHU-KZAA-IK-2/05},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/TechReports/ResearchRepProperties.pdf}
}
|
| Santana R, Larrañaga P and Lozano JA (2005), "Interactions and dependencies in estimation of distribution algorithms", In Proceedings of the 2005 Congress on Evolutionary Computation CEC-2005. Edinburgh, U.K. , pp. 1418-1425. IEEE Press. |
| Abstract: In this paper, we investigate two issues related to probabilistic modeling instimation of distribution algorithms (EDAs). First, we analyze the effect of selection in the arousal of probability dependencies in EDAs for random functions. We show that, for these functions, independence relationships not represented by the function structure are likely to appear in the probability model. Second, we propose an approach to approximate probability distributions in EDAs using a subset of the dependencies that exist in the data. An EDA that employs only malign interactions is introduced. Preliminary experiments presented show how the probability approximations based solely on malign interactions, can be applied to EDAs. |
BibTeX:
@inproceedings{Santana_et_al:2005b,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
title = {Interactions and dependencies in estimation of distribution algorithms},
booktitle = {Proceedings of the 2005 Congress on Evolutionary Computation CEC-2005},
publisher = {IEEE Press},
year = {2005},
pages = {1418--1425},
url = {http://dx.doi.org/10.1109/CEC.2005.1554856}
}
|
| Santana R, Larrañaga P and Lozano JA (2005), "Aprendizaje y muestreo de la aproximación Kikuchi", In Proceedings of the III Taller Nacional de Minería de Datos y Aprendizaje (TAMIDA-2005). Granada, Spain , pp. 97-105. Thomson. |
| Abstract: En este trabajo se presentan un algoritmo para el aprendizaje de la aproximación Kikuchi a partir de datos así como un método de muestreo de la referida aproximación. Se discuten resultados preliminares de la evaluación del algoritmo en el aprendizaje en datos generados a partir de una instancia del modelo Ising de ferromagnetismo. Los resultados obtenidos indican que la aproximación Kikuchi es capaz de representar de manera factible las dependencias existentes en los datos entre las distintas variables. |
BibTeX:
@inproceedings{Santana_et_al:2005c,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
title = {Aprendizaje y muestreo de la aproximación Kikuchi},
booktitle = {Proceedings of the III Taller Nacional de Minería de Datos y Aprendizaje (TAMIDA-2005)},
publisher = {Thomson},
year = {2005},
pages = {97--105},
url = {http://www.lsi.us.es/redmidas/CEDI/papers/501.pdf}
}
|
| Santana R, Larrañaga P and Lozano JA (2005), "Protein structure prediction in simplified models with estimation of distribution algorithms", In Proceedings of the IV Congreso Español sobre Metaheurísticas, Algoritmos Evolutivos y Bioinspirados (MAEB-2005). Granada, Spain , pp. 245-252. Thomson. |
| Abstract: In this paper we discuss the use of probabilistic modeling in the solution of the protein structure prediction problem. Estimation of distribution algorithms (EDAs) based on Markov models are presented as an alternative to other nature-inspired optimization algorithms for the solution of protein simplified models. |
BibTeX:
@inproceedings{Santana_et_al:2005d,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
title = {Protein structure prediction in simplified models with estimation of distribution algorithms},
booktitle = {Proceedings of the IV Congreso Español sobre Metaheurísticas, Algoritmos Evolutivos y Bioinspirados (MAEB-2005)},
publisher = {Thomson},
year = {2005},
pages = {245--252}
}
|
| Santana R, Larrañaga P and Lozano JA (2006), "Mixtures of Kikuchi approximations", In Proceedings of the 17th European Conference on Machine Learning: ECML 2006. Vol. 4212, pp. 365-376. Springer. |
| Abstract: Mixtures of distributions concern modeling a probability distribution by a weighted sum of other distributions. Kikuchi approximations of probability distributions follow an approach to approximate the free energy of statistical systems. In this paper, we introduce the mixture of Kikuchi approximations as a probability model. We present an algorithm for learning Kikuchi approximations from data based on the expectation-maximization (EM) paradigm. The proposal is tested in the approximation of probability distributions that arise in evolutionary computation. |
BibTeX:
@inproceedings{Santana_et_al:2006a,
author = {R. Santana and Pedro Larrañaga and J. A. Lozano},
editor = {Johannes Fürnkranz and Tobias Scheffer and Myra Spiliopoulou},
title = {Mixtures of Kikuchi approximations},
booktitle = {Proceedings of the 17th European Conference on Machine Learning: ECML 2006},
publisher = {Springer},
year = {2006},
volume = {4212},
pages = {365-376},
url = {http://dx.doi.org/10.1007/11871842_36}
}
|
| Santana R, Larrañaga P and Lozano JA (2007), "Side Chain Placement Using Estimation of Distribution Algorithms", Artificial Intelligence in Medicine. Vol. 39(1), pp. 49-63. |
| Abstract: Objective This paper presents an algorithm for the solution of the side chain placement problem. Methods and materials: The algorithm combines the application of the Goldstein elimination criterion with the univariate marginal distribution algorithm (UMDA), which stochastically searches the space of possible solutions. The suitability of the algorithm to address the problem is investigated using a set of proteins. Results: For a number of difficult instances where inference algorithms do not converge, it has been shown that UMDA is able to find better structures. Conclusions: The results obtained show that the algorithm can achieve better structures than those obtained with other state-of-the-art methods like inference-based techniques. Additionally, a theoretical and empirical analysis of the computational cost of the algorithm introduced has been presented. |
BibTeX:
@article{Santana_et_al:2007,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
title = {Side Chain Placement Using Estimation of Distribution Algorithms},
journal = {Artificial Intelligence in Medicine},
year = {2007},
volume = {39},
number = {1},
pages = {49--63},
url = {http://dx.doi.org/10.1016/j.artmed.2006.04.004}
}
|
| Santana R, Larrañaga P and Lozano JA (2007), "The role of a priori information in the minimization of contact potentials by means of estimation of distribution algorithms", In Proceedings of the Fifth European Conference on Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics. Valencia, Spain Vol. 4447, pp. 247-257. Springer. |
| Abstract: Directed search methods and probabilistic approaches have been used as two alternative ways for computational protein design. This paper presents a hybrid methodology that combines features from both approaches. Three estimation of distribution algorithms are applied to the solution of a protein design problem by minimization of contact potentials. The combination of probabilistic models able to represent probabilistic dependencies with the use of information about residues interactions in the protein contact graph is shown to improve the efficiency of search for the problems evaluated. |
BibTeX:
@inproceedings{Santana_et_al:2007b,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
editor = {E. Marchiori and J. H. Moore and J. C. Rajapakse},
title = {The role of a priori information in the minimization of contact potentials by means of estimation of distribution algorithms},
booktitle = {Proceedings of the Fifth European Conference on Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics},
publisher = {Springer},
year = {2007},
volume = {4447},
pages = {247-257},
url = {http://dx.doi.org/10.1007/978-3-540-71783-6_24}
}
|
| Santana R, Larrañaga P and Lozano JA (2007), "Challenges and open problems in discrete EDAs". Research Report at: Department of Computer Science and Artificial Intelligence, University of the Basque Country., October, 2007. (EHU-KZAA-IK-1/07) |
| Abstract: In this paper, we treat the identification of some of the problems that are relevant for the improvement and development of estimation of distribution algorithms. We present a survey of current challenges where further research must provide answers that extend the potential and applicability of the algorithms. In each case we state the problem and elaborate on the reasons that make it relevant for estimation of distribution algorithms. In some cases current work or possible alternatives for the solution of the problem are discussed. |
BibTeX:
@techreport{Santana_et_al:2007f,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
title = {Challenges and open problems in discrete EDAs},
school = {Department of Computer Science and Artificial Intelligence, University of the Basque Country},
year = {2007},
number = {EHU-KZAA-IK-1/07},
url = {https://www.researchgate.net/publication/228927458_Challenges_and_open_problems_in_discrete_EDAs}
}
|
| Santana R, Lozano JA and Larrañaga P (2007), "Algoritmos de Estimación de Distribuciones para el problema de la determinación de la cadena lateral de una proteína", In Memorias del V Congreso Español de Algoritmos Evolutivos y Bioinspirados. Tenerife, Spain , pp. 663-670. |
| Abstract: Este trabajo analiza diferentes mejoras a una aplicación precedente de los algoritmos de estimación de distribuciones (EDA) al problema de la determinación de la cadena lateral de una proteína. EDAs simples como el UMDA han permitido la obtención de cadenas laterales con menor valor de energía que las obtenidas con otros métodos para diferentes secuencias. Sin embargo para algunas secuencias, los resultados obtenidos con el UMDA no mejoran aquellos obtenidos por otros métodos. Por lo tanto, un problema de interés consiste en el estudio de métodos para aumentar la eficacia de los EDAs en la obtención de cadenas laterales de proteínas. Presentamos dos posibles alternativas utilizadas con este propósito: el uso de algoritmos de optimización local y el aprendizaje de modelos probabilíticos que tienen en cuenta las interacciones entre las variables del problema. Los algoritmos introducidos son evaluados en un conjunto de instancias difíles. Los resultados obtenidos en este conjunto son superiores a los alcanzados con algoritmos de optimización basados en inferencia |
BibTeX:
@inproceedings{Santana_et_al:2007g,
author = {R. Santana and and J. A. Lozano and P. Larrañaga},
title = {Algoritmos de Estimación de Distribuciones para el problema de la determinación de la cadena lateral de una proteína},
booktitle = {Memorias del V Congreso Español de Algoritmos Evolutivos y Bioinspirados},
year = {2007},
pages = {663-670}
}
|
| Santana R, Larrañaga P and Lozano JA (2008), "Estimation of distribution algorithms with affinity propagation methods". Research Report at: Department of Computer Science and Artificial Intelligence, University of the Basque Country., January, 2008. (EHU-KZAA-IK-1/08) |
| Abstract: Estimation of distribution algorithms (EDAs) that use marginal product model factorizations have been widely applied to a broad range of, mainly binary, optimization problems.In this paper, we introduce the affinity propagation EDA which learns a marginal product model by clustering a matrix of mutual information learned from the data using a very efficient message-passing algorithm known as affinity propagation. The introduced algorithm is tested on a set of binary and non-binary decomposable functions and using a hard combinatorial class of problem known as the HP protein model. The results show that the algorithm is a very efficient alternative to other EDAs that use marginal product model factorizations such as the extended compact genetic algorithm (ECGA) and improves the quality of the results achieved by ECGA when the cardinality of the variables is increased. |
BibTeX:
@techreport{Santana_et_al:2008,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
title = {Estimation of distribution algorithms with affinity propagation methods},
school = {Department of Computer Science and Artificial Intelligence, University of the Basque Country},
year = {2008},
number = {EHU-KZAA-IK-1/08},
url = {https://www.researchgate.net/publication/220375120_Learning_Factorizations_in_Estimation_of_Distribution_Algorithms_Using_Affinity_Propagation}
}
|
| Santana R, Larrañaga P and Lozano JA (2008), "Protein folding in simplified models with estimation of distribution algorithms", IEEE Transactions on Evolutionary Computation. Vol. 12(4), pp. 418-438. |
| Abstract: Simplified lattice models have played an important role in protein structure prediction and protein folding problems. These models can be useful for an initial approximation of the protein structure, and for the investigation of the dynamics that govern the protein folding process. Estimation of distribution algorithms (EDAs) are efficient evolutionary algorithms that can learn and exploit the search space regularities in the form of probabilistic dependencies. This paper introduces the application of different variants of EDAs to the solution of the protein structure prediction problem in simplified models, and proposes their use as a simulation tool for the analysis of the protein folding process. We develop new ideas for the application of EDAs to the bidimensional and tridimensional (2-d and 3-d) simplified protein folding problems. This paper analyzes the rationale behind the application of EDAs to these problems, and elucidates the relationship between our proposal and other population-based approaches proposed for the protein folding problem. We argue that EDAs are an efficient alternative for many instances of the protein structure prediction problem and are indeed appropriate for a theoretical analysis of search procedures in lattice models. All the algorithms introduced are tested on a set of difficult 2-d and 3-d instances from lattice models. Some of the results obtained with EDAs are superior to the ones obtained with other well-known population-based optimization algorithms. |
BibTeX:
@article{Santana_et_al:2008a,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
title = {Protein folding in simplified models with estimation of distribution algorithms},
journal = {IEEE Transactions on Evolutionary Computation},
year = {2008},
volume = {12},
number = {4},
pages = {418--438},
url = {http://dx.doi.org/10.1109/TEVC.2007.906095}
}
|
| Santana R, Larrañaga P and Lozano JA (2008), "Component weighting functions for adaptive search with EDAs", In Proceedings of the 2008 Congress on Evolutionary Computation CEC-2008. Hong Kong , pp. 4067-4074. IEEE Press. |
| Abstract: This paper introduces the component weighting approach as a general optimization heuristic to increase the likelihood of escaping from local optima by dynamically modifying the fitness function. The approach is tested on the optimization of the simplified hydrophobic-polar (HP) protein problem using estimation of distribution algorithms (EDAs). We show that the use of component weighting together with statistical information extracted from the set of selected solutions considerably improve the results of EDAs for the HP problem. The paper also elaborates on the use of probabilistic modeling for the definition of dynamic fitness functions and on the use of combinations of models. |
BibTeX:
@inproceedings{Santana_et_al:2008b,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
title = {Component weighting functions for adaptive search with EDAs},
booktitle = {Proceedings of the 2008 Congress on Evolutionary Computation CEC-2008},
publisher = {IEEE Press},
year = {2008},
pages = {4067--4074},
url = {http://dx.doi.org/10.1109/CEC.2008.4631352}
}
|
| Santana R, Larrañaga P and Lozano JA (2008), "Adaptive estimation of distribution algorithms", In Adaptive and Multilevel Metaheuristics. Vol. 136, pp. 177-197. Springer. |
| Abstract: Estimation of distribution algorithms (EDAs) are evolutionary methods that use probabilistic models instead of genetic operators to lead the search. Most of current proposals on EDAs do not incorporate adaptive techniques. Usually, the class of probabilistic model employed as well as the learning and sampling methods are static. In this paper, we present a general framework for introducing adaptation in EDAs. This framework allows the possibility of changing the class of probabilistic models during the evolution. We present a number of measures, and techniques that can be used to evaluate the effect of the EDA components in order to design adaptive EDAs. As a case of study we present an adaptive EDA that combines different classes of probabilistic models and sampling methods. The algorithm is evaluated in the solution of the satisfiability problem.. |
BibTeX:
@incollection{Santana_et_al:2008c,
author = {R. Santana and Pedro Larrañaga and J. A. Lozano},
editor = {C. Cotta and M. Sevaux and K. Sörensen},
title = {Adaptive estimation of distribution algorithms},
booktitle = {Adaptive and Multilevel Metaheuristics},
publisher = {Springer},
year = {2008},
volume = {136},
pages = {177-197},
url = {http://dx.doi.org/10.1007/978-3-540-79438-7_9}
}
|
| Santana R, Larrañaga P and Lozano JA (2008), "Adding probabilistic dependencies to the search of protein side chain configurations using EDAs", In Parallel Problem Solving from Nature - PPSN X. Dortmund, Germany Vol. 5199, pp. 1120-1129. Springer. |
| Abstract: The problem of finding an optimal positioning for the side chain residues of a protein is called the side chain placement or side chain prediction problem. It can be posed as an optimization problem in the discrete domain. In this paper we use an estimation of distribution algorithm to address this optimization problem. Using a set of 50 difficult protein instances, it is shown that the addition of dependencies between the variables in the probabilistic model can improve the quality of the solutions achieved for most of the instances considered. However, we also show that only when information about the known interactions between the residues is considered in the creation of the probabilistic model, the addition of the dependencies contributes to improve the quality of the solutions obtained. |
BibTeX:
@inproceedings{Santana_et_al:2008d,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
editor = {G. Rudolph and T. Jansen and S. Lucas and C. Poloni and N. Beume},
title = {Adding probabilistic dependencies to the search of protein side chain configurations using EDAs},
booktitle = {Parallel Problem Solving from Nature - PPSN X},
publisher = {Springer},
year = {2008},
volume = {5199},
pages = {1120--1129},
url = {http://dx.doi.org/10.1007/978-3-540-87700-4_111}
}
|
| Santana R, Larrañaga P and Lozano JA (2008), "Combining Variable Neighborhood Search and Estimation of Distribution Algorithms in the Protein Side Chain Placement Problem", Journal of Heuristics. Vol. 14, pp. 519-547. |
| Abstract: The aim of this work is to introduce several proposals for combining two metaheuristics: variable neighborhood search (VNS) and estimation of distribution algorithms (EDAs). Although each of these metaheuristics has been previously hybridized in several ways, this paper constitutes the first attempt to combine both optimization methods. The different ways of combining VNS and EDAs will be classified into three groups. In the first group, we will consider combinations where the philosophy underlying VNS is embedded in EDAs. Considering different neighborhood spaces (points, populations or probability distributions), we will obtain instantiations for the approaches in this group. The second group of algorithms is obtained when probabilistic models (or any other machine learning paradigm) are used in order to exploit the good and bad shakes of the randomly generated solutions in a reduced variable neighborhood search. The last group of algorithms contains the results of alternating VNS and EDAs. An application of the first approach is presented in the protein side chain placement problem. The results obtained show the superiority of the hybrid algorithm in comparison with EDAs and VNS. |
BibTeX:
@article{Santana_et_al:2008f,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
title = {Combining Variable Neighborhood Search and Estimation of Distribution Algorithms in the Protein Side Chain Placement Problem},
journal = {Journal of Heuristics},
year = {2008},
volume = {14},
pages = {519--547},
url = {http://dx.doi.org/10.1007/s10732-007-9049-8}
}
|
| Santana R, Mendiburu A and Lozano JA (2008), "An empirical analysis of loopy belief propagation in three topologies: Grids, small-world networks and random graphs", In Proceedings of the Fourth European Workshop on Probabilistic Graphical Models (PGM-2008). Hirtshals, Denmark , pp. 249-256. |
| Abstract: Recently, much research has been devoted to the study of loopy belief propagation algorithm. However, little attention has been paid to the change of its behavior in relation with the problem graph topology. In this paper we empirically study the behavior of loopy belief propagation on different network topologies which include grids, small-world networks and random graphs. In our experiments, several descriptors of the algorithm are collected in order to analyze its behavior. We show that the performance of the algorithm is highly sensitive to changes in the topologies. Furthermore, evidence is given showing that the addition of shortcuts to grids can determine important changes in the dynamics of the algorithm. |
BibTeX:
@inproceedings{Santana_et_al:2008g,
author = {R. Santana and A. Mendiburu and J. A. Lozano},
editor = {Manfred Jaeger and Thomas D. Nielsen},
title = {An empirical analysis of loopy belief propagation in three topologies: Grids, small-world networks and random graphs},
booktitle = {Proceedings of the Fourth European Workshop on Probabilistic Graphical Models (PGM-2008)},
year = {2008},
pages = {249--256}
}
|
| Santana R, Bielza C, Lozano JA and Larrañaga P (2009), "Mining probabilistic models learned by EDAs in the optimization of multi-objective problems", In Proceedings of the Genetic and Evolutionary Computation Conference GECCO-2009. New York, NY, USA , pp. 445-452. ACM. |
| Abstract: One of the uses of the probabilistic models learned by estimation of distribution algorithms is to reveal previous unknown information about the problem structure. In this paper we investigate the mapping between the problem structure and the dependencies captured in the probabilistic models learned by EDAs for a set of multi-objective satisfiability problems. We present and discuss the application of different data mining and visualization techniques for processing and visualizing relevant information from the structure of the learned probabilistic models. We show that also in the case of multi-objective optimization problems, some features of the original problem structure can be translated to the probabilistic models and unveiled by using algorithms that mine the model structures. |
BibTeX:
@inproceedings{Santana_et_al:2009,
author = {R. Santana and C. Bielza and J. A. Lozano and P. Larrañaga},
title = {Mining probabilistic models learned by EDAs in the optimization of multi-objective problems},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference GECCO-2009},
publisher = {ACM},
year = {2009},
pages = {445-452},
url = {http://dl.acm.org/citation.cfm?id=1569963}
}
|
| Santana R, Echegoyen C, Mendiburu A, Bielza C, Lozano JA, Larrañaga P, Armañanzas R and Shakya S (2009), "MATEDA: A suite of EDA programs in Matlab". Research Report at: Department of Computer Science and Artificial Intelligence, University of the Basque Country., February, 2009. (EHU-KZAA-IK-2/09) |
| Abstract: This paper describes MATEDA-2.0, a suite of programs in Matlab for estimation of distribution algorithms. The package allows the optimization of single and multi-objective problems with estimation of distribution algorithms (EDAs) based on undirected graphical models and Bayesian networks. The implementation is conceived for allowing the incorporation by the user of different combinations of selection, learning, sampling, and local search procedures. Other included methods allow the analysis of the structures learned by the probabilistic models, the visualization of particular features of these structures and the use of the probabilistic models as fitness modeling tools. |
BibTeX:
@techreport{Santana_et_al:2009a,
author = {R. Santana and C. Echegoyen and A. Mendiburu and C. Bielza and J. A. Lozano and P. Larrañaga and R. Armañanzas and S. Shakya},
title = {MATEDA: A suite of EDA programs in Matlab},
school = {Department of Computer Science and Artificial Intelligence, University of the Basque Country},
year = {2009},
number = {EHU-KZAA-IK-2/09},
url = {http://hdl.handle.net/10810/4622}
}
|
| Santana R, Larrañaga P and Lozano JA (2009), "Research topics on discrete estimation of distribution algorithms", Memetic Computing. Vol. 1(1), pp. 35-54. |
| Abstract: In this paper, we identify a number of topics relevant for the improvement and development of discrete estimation of distribution algorithms. Focusing on the role of probability distributions and factorizations in estimation of distribution algorithms, we present a survey of current challenges where further research must provide answers that extend the potential and applicability of these algorithms. In each case we state the research topic and elaborate on the reasons that make it relevant for estimation of distribution algorithms. In some cases current work or possible alternatives for the solution of the problem are discussed. |
BibTeX:
@article{Santana_et_al:2009b,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
title = {Research topics on discrete estimation of distribution algorithms},
journal = {Memetic Computing},
year = {2009},
volume = {1},
number = {1},
pages = {35-54},
url = {http://dx.doi.org/10.1007/s12293-008-0002-7}
}
|
| Santana R, Larrañaga P and Lozano JA (2010), "Learning factorizations in estimation of distribution algorithms using affinity propagation", Evolutionary Computation. Vol. 18(4), pp. 515-546. |
| Abstract: Estimation of distribution algorithms (EDAs) that use marginal product model factorizations have been widely applied to a broad range of mainly binary optimization problems. In this paper, we introduce the affinity propagation EDA (AffEDA) which learns a marginal product model by clustering a matrix of mutual information learned from the data using a very efficient message-passing algorithm known as affinity propagation. The introduced algorithm is tested on a set of binary and nonbinary decomposable functions and using a hard combinatorial class of problem known as the HP protein model. The results show that the algorithm is a very efficient alternative to other EDAs that use marginal product model factorizations such as the extended compact genetic algorithm (ECGA) and improves the quality of the results achieved by ECGA when the cardinality of the variables is increased. |
BibTeX:
@article{Santana_et_al:2009c,
author = {R. Santana and P. Larrañaga and J. A. Lozano},
title = {Learning factorizations in estimation of distribution algorithms using affinity propagation},
journal = {Evolutionary Computation},
year = {2010},
volume = {18},
number = {4},
pages = {515-546},
url = {http://www.mitpressjournals.org/doi/abs/10.1162/EVCO_a_00002}
}
|
| Santana R, Mendiburu A, Zaitlen N, Eskin E and Lozano JA (2009), "On the application of estimation of distribution algorithms to multi-marker tagging SNP selection". Research Report at: Department of Computer Science and Artificial Intelligence, University of the Basque Country., July, 2009. (EHU-KZAA-IK-4/09) |
| Abstract: This paper presents an algorithm for the automatic selection of a minimal subset of tagging single nucleotide polymorphisms (SNPs) using an estimation of distribution algorithm (EDA). The EDA stochastically searches the constrained space of possible feasible solutions and takes advantage of the underlying topological structure defined by the SNP correlations to model the problem interactions. The algorithm is evaluated across the HapMap reference panel data sets. The introduced algorithm is effective for the identification of minimal multi-marker SNP sets, which considerably reduce the dimension of the tagging SNP set in comparison with single-marker sets. New reduced tagging sets are obtained for all the HapMap SNP regions considered. We also show that the information extracted from the interaction graph representing the correlations between the SNPs can help to improve the efficiency of the optimization algorithm. |
BibTeX:
@techreport{Santana_et_al:2009d,
author = {R. Santana and A. Mendiburu and N. Zaitlen and E. Eskin and J. A. Lozano},
title = {On the application of estimation of distribution algorithms to multi-marker tagging SNP selection},
school = {Department of Computer Science and Artificial Intelligence, University of the Basque Country},
year = {2009},
number = {EHU-KZAA-IK-4/09},
url = {https://addi.ehu.es/bitstream/handle/10810/4623/tr09-00-4.pdf?sequence=1&isAllowed=y}
}
|
| Santana R, Bielza C, Larrañaga P, Lozano JA, Echegoyen C, Mendiburu A, Armañanzas R and Shakya S (2010), "Mateda-2.0: A MATLAB package for the implementation and analysis of estimation of distribution algorithms", Journal of Statistical Software. Vol. 35(7), pp. 1-30. American Statistical Association. |
| Abstract: This paper describes Mateda-2.0, a MATLAB package for estimation of distribution algorithms (EDAs). This package can be used to solve single and multi-objective discrete and continuous optimization problems using EDAs based on undirected and directed probabilistic graphical models. The implementation contains several methods commonly employed by EDAs. It is also conceived as an open package to allow users to incorporate different combinations of selection, learning, sampling, and local search procedures. Additionally, it includes methods to extract, process and visualize the structures learned by the probabilistic models. This way, it can unveil previously unknown information about the optimization problem domain. Mateda-2.0 also incorporates a module for creating and validating function models based on the probabilistic models learned by EDAs. |
BibTeX:
@article{Santana_et_al:2009h,
author = {R. Santana and C. Bielza and P. Larrañaga and J. A. Lozano and C. Echegoyen and A. Mendiburu and R. Armañanzas and S. Shakya},
title = {Mateda-2.0: A MATLAB package for the implementation and analysis of estimation of distribution algorithms},
journal = {Journal of Statistical Software},
publisher = {American Statistical Association},
year = {2010},
volume = {35},
number = {7},
pages = {1-30},
url = {http://www.jstatsoft.org/v35/i07}
}
|
| Santana R, Bielza C and Larrañaga P (2010), "Synergies between network-based representations and probabilistic graphical modeling in the solution of problems from neuroscience", In Proceedings of the Twenty Third International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems. Córdoba, España Vol. 6098, pp. 149-158. Springer. |
| Abstract: Neural systems network-based representations are useful tools to analyze numerous phenomena in neuroscience. Probabilistic graphical models (PGMs) give a concise and still rich representation of complex systems from different domains, including neural systems. In this paper we analyze the characteristics of a bidirectional relationship between networks-based representations and PGMs. We show the way in which this relationship can be exploited introducing a number of methods for the solution of classification, inference and optimization problems. To illustrate the applicability of the introduced methods, a number of problems from the field of neuroscience, in which ongoing research is conducted, are used. |
BibTeX:
@inproceedings{Santana_et_al:2010a,
author = {R. Santana and Concha Bielza and Pedro Larrañaga},
editor = {N. García-Pedrajas et. al.},
title = {Synergies between network-based representations and probabilistic graphical modeling in the solution of problems from neuroscience},
booktitle = {Proceedings of the Twenty Third International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems},
publisher = {Springer},
year = {2010},
volume = {6098},
pages = {149-158},
url = {http://dx.doi.org/10.1007/978-3-642-13033-5_16}
}
|
| Santana R, Bielza C and Larrañaga P (2010), "Using probabilistic dependencies improves the search of conductance-based compartmental neuron models", In Proceedings of the 8th European Conference on Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics. Istanbul, Turkey Vol. 6023, pp. 170-181. Springer. |
| Abstract: Conductance-based compartmental neuron models are traditionally used to investigate the electrophysiological properties of neurons. These models require a number of parameters to be adjusted to biological experimental data and this question can be posed as an optimization problem. In this paper we investigate the behavior of different estimation of distribution algorithms (EDAs) for this problem. We focus on studying the influence that the interactions between the neuron model conductances have in the complexity of the optimization problem. We support evidence that the use of these interactions during the optimization process can improve the EDA behavior. |
BibTeX:
@inproceedings{Santana_et_al:2010c,
author = {R. Santana and C. Bielza and P. Larrañaga},
editor = {Clara Pizzuti and Marylyn D. Ritchie and Mario Giacobini},
title = {Using probabilistic dependencies improves the search of conductance-based compartmental neuron models},
booktitle = {Proceedings of the 8th European Conference on Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics},
publisher = {Springer},
year = {2010},
volume = {6023},
pages = {170-181},
url = {http://dx.doi.org/10.1007/978-3-642-12211-8_15}
}
|
| Santana R, Mendiburu A, Zaitlen N, Eskin E and Lozano JA (2010), "Multi-marker tagging single nucleotide polymorphism selection using estimation of distribution algorithms", Artificial Intelligence in Medicine. Vol. 50, pp. 193-201. |
| Abstract: Objectives This paper presents an optimization algorithm for the automatic selection of a minimal subset of tagging single nucleotide polymorphisms (SNPs). Methods and materials: The determination of the set of minimal tagging SNPs is approached as an optimization problem in which each tagged SNP can be covered by a single tagging SNP or by a pair of tagging SNPs. The problem is solved using an estimation of distribution algorithm (EDA) which takes advantage of the underlying topological structure defined by the SNP correlations to model the problem interactions. The EDA stochastically searches the constrained space of feasible solutions. It is evaluated across HapMap reference panel data sets. Results: The EDA was compared with a SAT solver, able to find the single-marker minimal tagging sets, and with the Tagger program. The percentage of reduction ranged from 10 to 43 percentage in the number of tagging SNPs of the minimal multi-marker tagging set found by the EDA with respect to the other algorithms. Conclusions: The introduced algorithm is effective for the identification of minimal multi-marker SNP sets, which considerably reduce the dimension of the tagging SNP set in comparison with single-marker sets. Other variants of the SNP problem can be treated following the same approach. |
BibTeX:
@article{Santana_et_al:2010d,
author = {R. Santana and A. Mendiburu and N. Zaitlen and E. Eskin and J. A. Lozano},
title = {Multi-marker tagging single nucleotide polymorphism selection using estimation of distribution algorithms},
journal = {Artificial Intelligence in Medicine},
year = {2010},
volume = {50},
pages = {193-201},
url = {http://www.sciencedirect.com/science/article/pii/S0933365710000758}
}
|
| Santana R, Bielza C and Larrañaga P (2010), "Network measures for re-using problem information in EDAs". Research Report at: Department of Artificial Intelligence, Faculty of Informatics, Technical University of Madrid., June, 2010. (UPM-FI/DIA/2010-3) |
| Abstract: Probabilistic graphical models (PGMs) are used in estimation of distribution algorithms (EDAs) as a model of the search space. Graphical components of PGMs can be also analyzed as networks. In this paper we show that topological measures extracted from these networks capture characteristic information of the optimization problem. The measures can be also used to describe the EDA behavior. Using a simplified protein folding optimization problem, we show that the network information extracted from a set of problem instances can be effectively used to predict characteristics of similar instances. |
BibTeX:
@techreport{Santana_et_al:2010e,
author = {R. Santana and C. Bielza and P. Larrañaga},
title = {Network measures for re-using problem information in EDAs},
school = {Department of Artificial Intelligence, Faculty of Informatics, Technical University of Madrid},
year = {2010},
number = {UPM-FI/DIA/2010-3},
url = {https://www.researchgate.net/publication/228738540_Network_measures_for_re-using_problem_information_in_EDAs}
}
|
| Santana R, Muelas S, Latorre A and Peña JM (2011), "A direct optimization approach to the P300 speller", In Proceedings of the 2011 Genetic and Evolutionary Computation Conference GECCO-2011. Dublin, Ireland , pp. 1747-1754. |
| Abstract: The P300 component of the brain event-related-potential is one of the most used signals in brain computer interfaces (BCIs). One of the required steps for the application of the P300 paradigm is the identification of this component in the presence of stimuli. In this paper we propose a direct optimization approach to the P300 classification problem. A general formulation of the problem is introduced. Different classes of optimization algorithms are applied to solve the problem and the concepts of k-best and k-worst ensembles of solutions are introduced as a way to improve the accuracy of single solutions. The introduced approaches are able to achieve a classification rate over 80 percentage on test data. |
BibTeX:
@inproceedings{Santana_et_al:2011,
author = {R. Santana and S. Muelas and A. Latorre and J. M. Peña},
title = {A direct optimization approach to the P300 speller},
booktitle = {Proceedings of the 2011 Genetic and Evolutionary Computation Conference GECCO-2011},
year = {2011},
pages = {1747-1754},
url = {http://dl.acm.org/citation.cfm?id=2001811}
}
|
| Santana R, Bielza C and Larrañaga P (2011), "Optimizing brain networks topologies using multi-objective evolutionary computation", Neuroinformatics. Vol. 9(1), pp. 3-19. |
| Abstract: The analysis of brain network topological features has served to better understand these networks and reveal particular characteristics of their functional behavior. The distribution of brain network motifs is particularly useful for detecting and describing differences between brain networks and random and computationally optimized artificial networks. In this paper we use a multi-objective evolutionary optimization approach to generate optimized artificial networks that have a number of topological features resembling brain networks. The Pareto set approximation of the optimized networks is used to extract network descriptors that are compared to brain and random network descriptors. To analyze the networks, the clustering coefficient, the average path length, the modularity and the betweenness centrality are computed. We argue that the topological complexity of a brain network can be estimated using the number of evaluations needed by an optimization algorithm to output artificial networks of similar complexity. For the analyzed network examples, our results indicate that while original brain networks have a reduced structural motif number and a high functional motif number, they are not optimal with respect to these two topological features. We also investigate the correlation between the structural and functional motif numbers, the average path length and the clustering coefficient in random, optimized and brain networks. |
BibTeX:
@article{Santana_et_al:2011a,
author = {R. Santana and C. Bielza and P. Larrañaga},
title = {Optimizing brain networks topologies using multi-objective evolutionary computation},
journal = {Neuroinformatics},
year = {2011},
volume = {9},
number = {1},
pages = {3-19},
url = {http://dx.doi.org/10.1007/s12021-010-9085-7}
}
|
| Santana R, Bielza C and Larrañaga P (2011), "Affinity propagation enhanced by estimation of distribution algorithms", In Proceedings of the 2011 Genetic and Evolutionary Computation Conference GECCO-2011. Dublin, Ireland , pp. 331-338. |
| Abstract: Tumor classification based on gene expression data can be applied to set appropriate medical treatment according to the specific tumor characteristics. In this paper we propose the use of estimation of distribution algorithms (EDAs) to enhance the performance of affinity propagation (AP) in classification problems. AP is an efficient clustering algorithm based on message-passing methods and which automatically identifies exemplars of each cluster. We introduce an EDA-based procedure to compute the preferences used by the AP algorithm. Our results show that AP performance can be notably improved by using the introduced approach. Furthermore, we present evidence that classification of new data is improved by employing previously identified exemplars with only minor decrease in classification accuracy. |
BibTeX:
@inproceedings{Santana_et_al:2011b,
author = {R. Santana and C. Bielza and P. Larrañaga},
title = {Affinity propagation enhanced by estimation of distribution algorithms},
booktitle = {Proceedings of the 2011 Genetic and Evolutionary Computation Conference GECCO-2011},
year = {2011},
pages = {331-338},
url = {http://dl.acm.org/citation.cfm?id=2001622}
}
|
| Santana R, Karshenas H, Bielza C and Larrañaga P (2011), "Regularized k-order Markov models in EDAs", In Proceedings of the 2011 Genetic and Evolutionary Computation Conference GECCO-2011. Dublin, Ireland , pp. 593-600. |
| Abstract: k-order Markov models have been introduced to estimation of distribution algorithms (EDAs) to solve a particular class of optimization problems in which each variable depends on its previous k variables in a given, fixed order. In this paper we investigate the use of regularization as a way to approximate k-order Markov models when k is increased. The introduced regularized models are used to balance the complexity and accuracy of the k-order Markov models. We investigate the behavior of the EDAs in several instances of the hydrophobic-polar (HP) protein problem, a simplified protein folding model. Our preliminary results show that EDAs that use regularized approximations of the k-order Markov models offer a good compromise between complexity and efficiency, and could be an appropriate choice when the number of variables is increased. |
BibTeX:
@inproceedings{Santana_et_al:2011c,
author = {R. Santana and H. Karshenas and C. Bielza and P. Larrañaga},
title = {Regularized k-order Markov models in EDAs},
booktitle = {Proceedings of the 2011 Genetic and Evolutionary Computation Conference GECCO-2011},
year = {2011},
pages = {593-600},
url = {http://dl.acm.org/citation.cfm?id=2001658}
}
|
| Santana R, Karshenas H, Bielza C and Larrañaga P (2011), "Quantitative genetics in multi-objective optimization algorithms: From useful insights to effective methods", In Proceedings of the 2011 Genetic and Evolutionary Computation Conference GECCO-2011. Dublin, Ireland , pp. 91-92. |
| Abstract: This paper shows that statistical algorithms proposed for the quantitative trait loci (QTL) mapping problem, and the equation of the multivariate response to selection can be of application in multi-objective optimization. We introduce the conditional dominance relationships between the objectives and propose the use of results from QTL analysis and G-matrix theory to the analysis of multi-objective evolutionary algorithms (MOEAs). |
BibTeX:
@inproceedings{Santana_et_al:2011d,
author = {R. Santana and H. Karshenas and C. Bielza and P. Larrañaga},
title = {Quantitative genetics in multi-objective optimization algorithms: From useful insights to effective methods},
booktitle = {Proceedings of the 2011 Genetic and Evolutionary Computation Conference GECCO-2011},
year = {2011},
pages = {91-92},
url = {http://dl.acm.org/citation.cfm?id=2001911}
}
|
| Santana R, Bielza C and Larrañaga P (2011), "An ensemble of classifiers approach with multiple sources of information", In Proceedings of ICANN/PASCAL2 Challenge: MEG Mind Reading. , pp. 25-30. Aalto University. |
| Abstract: This paper describes the main characteristics of our approach to the ICANN-2011 Mind reading from MEG - PASCAL Challenge. The distinguished features of our method are: 1) The use of different sources of information as input to the classifiers. We simultaneously use information coming from raw data, channels correlations, mutual information between channels, and channel interactions graphs as features for the classifiers. 2) The use of ensemble of classifiers based on regularized multi-logistic regression, regression trees, and an affinity propagation based classifier. |
BibTeX:
@inproceedings{Santana_et_al:2011f,
author = {R. Santana and C. Bielza and P. Larrañaga},
editor = {A. Klami},
title = {An ensemble of classifiers approach with multiple sources of information},
booktitle = {Proceedings of ICANN/PASCAL2 Challenge: MEG Mind Reading},
publisher = {Aalto University},
year = {2011},
pages = {25--30},
url = {http://www.cis.hut.fi/icann2011/meg/megicann_santanaetal.pdf}
}
|
| Santana R, Bielza C and Larrañaga P (2012), "Regularized logistic regression and multi-objective variable selection for classifying MEG data", Biological Cybernetics. Vol. 106(6-7), pp. 389-405. |
| Abstract: This paper addresses the question of maximizing classifier accuracy for classifying task-related mental activity from Magnetoencelophalography (MEG) data. We propose the use of different sources of information and introduce an automatic channel selection procedure. To determine an informative set of channels, our approach combines a variety of machine learning algorithms: feature subset selection methods, classifiers based on regularized logistic regression, information fusion, and multiobjective optimization based on probabilistic modeling of the search space. The experimental results show that our proposal is able to improve classification accuracy compared to approaches whose classifiers use only one type of MEG information or for which the set of channels is fixed a priori. |
BibTeX:
@article{Santana_et_al:2012,
author = {R. Santana and C. Bielza and P. Larrañaga},
title = {Regularized logistic regression and multi-objective variable selection for classifying MEG data},
journal = {Biological Cybernetics},
year = {2012},
volume = {106},
number = {6-7},
pages = {389-405},
url = {http://dx.doi.org/10.1007/s00422-012-0506-6}
}
|
| Santana R, Bonnet L, Légeny J and Lécuyer A (2012), "Introducing the use of model-based evolutionary algorithms for EEG-based motor imagery classification", In Proceedings of the 2012 Genetic and Evolutionary Computation Conference GECCO-2012. Philadelphia, US , pp. 1159-1166. ACM Press. |
| Abstract: Brain computer interfaces (BCIs) allow the direct human-computer interaction without the need of motor intervention. To properly and efficiently decode brain signals into computer commands the application of machine-learning techniques is required. Evolutionary algorithms have been increasingly applied in different steps of BCI implementations. In this paper we introduce the use of the covariance matrix adaptation evolution strategy (CMA-ES) for BCI systems based on motor imagery. The optimization algorithm is used to evolve linear classifiers able to outperform other traditional classifiers. We also analyze the role of modeling variables interactions for additional insight in the understanding of the BCI paradigms. |
BibTeX:
@inproceedings{Santana_et_al:2012c,
author = {R. Santana and L. Bonnet and J. Légeny and A. Lécuyer},
title = {Introducing the use of model-based evolutionary algorithms for EEG-based motor imagery classification},
booktitle = {Proceedings of the 2012 Genetic and Evolutionary Computation Conference GECCO-2012},
publisher = {ACM Press},
year = {2012},
pages = {1159--1166},
url = {http://dl.acm.org/citation.cfm?id=2330323}
}
|
| Santana R, Bielza C and Larrañaga P (2012), "Maximizing the number of polychronous groups in spiking networks", In Companion Proceedings of the 2012 Genetic and Evolutionary Computation Conference GECCO-2012. Philadelphia, US , pp. 1499-1500. ACM Press. |
| Abstract: In this paper we investigate the effect of biasing the axonal connection delay values in the number of polychronous groups produced for a spiking neuron network model. We use an estimation of distribution algorithm (EDA) that learns tree models to search for optimal delay configurations. Our results indicate that the introduced approach can be used to considerably increase the number of such groups. |
BibTeX:
@inproceedings{Santana_et_al:2012d,
author = {R. Santana and C. Bielza and P. Larrañaga},
title = {Maximizing the number of polychronous groups in spiking networks},
booktitle = {Companion Proceedings of the 2012 Genetic and Evolutionary Computation Conference GECCO-2012},
publisher = {ACM Press},
year = {2012},
pages = {1499--1500},
url = {http://dl.acm.org/citation.cfm?id=2331012}
}
|
| Santana R, Mendiburu A and Lozano JA (2012), "Evolving NK-complexity for evolutionary solvers", In Companion Proceedings of the 2012 Genetic and Evolutionary Computation Conference GECCO-2012. Philadelphia, US , pp. 1473-1474. ACM Press. |
| Abstract: In this paper we empirically investigate the structural characteristics that can help to predict the complexity of NK-landscape instances for estimation of distribution algorithms (EDAs). We evolve instances that maximize the EDA complexity in terms of its success rate. Similarly, instances that minimize the algorithm complexity are evolved. We then identify network measures, computed from the structures of the NK-landscape instances, that have a statistically significant difference between the set of easy and hard instances. The features identified are consistently significant for different values of N and K. |
BibTeX:
@inproceedings{Santana_et_al:2012e,
author = {R. Santana and A. Mendiburu and J. A. Lozano},
title = {Evolving NK-complexity for evolutionary solvers},
booktitle = {Companion Proceedings of the 2012 Genetic and Evolutionary Computation Conference GECCO-2012},
publisher = {ACM Press},
year = {2012},
pages = {1473--1474},
url = {http://dl.acm.org/citation.cfm?id=2330997}
}
|
| Santana R, Mendiburu A and Lozano JA (2012), "Structural transfer using EDAs: An application to multi-marker tagging SNP selection", In Proceedings of the 2012 Congress on Evolutionary Computation CEC-2012. Brisbane, Australia , pp. 3484-3491. IEEE Press. |
| Abstract: In this paper we investigate the question of transfer learning in evolutionary optimization using estimation of distribution algorithms. We propose a framework for transfer learning between related optimization problems by means of structural transfer. Different methods for incrementing or replacing the (possibly unavailable) structural information of the target optimization problem are presented. As a test case we solve the multi-marker tagging single-nucleotide polymorphism (SNP) selection problem, a real world problem from genetics. The introduced variants of structural transfer are validated in the computation of tagging SNPs on a database of 1167 individuals from 58 human populations worldwide. Our experimental results show significant improvements over EDAs that do not incorporate information from related problems. |
BibTeX:
@inproceedings{Santana_et_al:2012f,
author = {R. Santana and A. Mendiburu and J. A. Lozano},
title = {Structural transfer using EDAs: An application to multi-marker tagging SNP selection},
booktitle = {Proceedings of the 2012 Congress on Evolutionary Computation CEC-2012},
publisher = {IEEE Press},
year = {2012},
pages = {3484-3491},
note = {Best Paper Award of 2012 Congress on Evolutionary Computation},
url = {http://dx.doi.org/10.1109/CEC.2012.6252963}
}
|
| Santana R, Mendiburu A and Lozano JA (2012), "An analysis of the use of probabilistic modeling for synaptic connectivity prediction from genomic data", In Proceedings of the 2012 Congress on Evolutionary Computation CEC-2012. Brisbane, Australia , pp. 3221-3228. IEEE Press. |
| Abstract: The identification of the specific genes that influence particular phenotypes is a common problem in genetic studies. In this paper we address the problem of determining the influence of gene joint expression in synapse predictability. The question is posed as an optimization problem in which the conditional entropy of gene subsets with respect to the synaptic connectivity phenotype is minimized. We investigate the use of single- and multi-objective estimation of distribution algorithms and focus on real data from C. elegans synaptic connectivity. We show that the introduced algorithms are able to compute gene sets that allow an accurate synapse predictability. However, the multi-objective approach can simultaneously search for gene sets with different number of genes. Our results also indicate that optimization problems defined on constrained binary spaces remain challenging for the conception of competitive estimation of distribution algorithm. |
BibTeX:
@inproceedings{Santana_et_al:2012g,
author = {R. Santana and A. Mendiburu and J. A. Lozano},
title = {An analysis of the use of probabilistic modeling for synaptic connectivity prediction from genomic data},
booktitle = {Proceedings of the 2012 Congress on Evolutionary Computation CEC-2012},
publisher = {IEEE Press},
year = {2012},
pages = {3221--3228},
url = {http://dx.doi.org/10.1109/CEC.2012.6252997}
}
|
| Santana R, Bielza C and Larrañaga P (2012), "Conductance interaction identification by means of Boltzmann distribution and mutual information analysis in conductance-based neuron models", BMC Neuroscience. Vol. 13(Suppl 1), pp. P100. BioMed Central. |
| Abstract: In this paper we propose a probabilistic approach based on the computation of the Boltzmann distribution and the mutual information of conductance interactions to learn higher-order, not necessarily pair-wise, potential co-regulation mechanisms from a database of the crustacean stomatogastric ganglion pyloric circuit models. |
BibTeX:
@article{Santana_et_al:2012h,
author = {Santana, R. and Bielza, C. and Larrañaga, P.},
title = {Conductance interaction identification by means of Boltzmann distribution and mutual information analysis in conductance-based neuron models},
journal = {BMC Neuroscience},
publisher = {BioMed Central},
year = {2012},
volume = {13},
number = {Suppl 1},
pages = {P100},
url = {http://dx.doi.org/10.1186/1471-2202-13-S1-P100}
}
|
| Santana R, Mendiburu A and Lozano JA (2012), "New methods for generating populations in Markov network based EDAs: Decimation strategies and model-based template recombination". Research Report at: Department of Computer Science and Artificial Intelligence, University of the Basque Country., December, 2012. (EHU-KZAA-TR:2012-05) |
| Abstract: Methods for generating a new population are a fundamental component of estimation of distribution algorithms (EDAs). They serve to transfer the information contained in the probabilistic model to the new generated population. In EDAs based on Markov networks, methods for generating new populations usually discard information contained in the model to gain in efficiency. Other methods like Gibbs sampling use information about all interactions in the model but are computationally very costly. In this paper we propose new methods for generating new solutions in EDAs based on Markov networks. We introduce approaches based on inference methods for computing the most probable configurations and model-based template recombination. We show that the application of different variants of inference methods can increase the EDAs convergence rate and reduce the number of function evaluations needed to find the optimum of binary and non-binary discrete functions. |
BibTeX:
@techreport{Santana_et_al:2012i,
author = {R. Santana and A. Mendiburu and J. A. Lozano},
title = {New methods for generating populations in Markov network based EDAs: Decimation strategies and model-based template recombination},
school = {Department of Computer Science and Artificial Intelligence, University of the Basque Country},
year = {2012},
number = {EHU-KZAA-TR:2012-05},
url = {http://hdl.handle.net/10810/9180}
}
|
| Santana R, Mendiburu A and Lozano JA (2012), "Using network mesures to test evolved NK-landscapes". Research Report at: Department of Computer Science and Artificial Intelligence, University of the Basque Country., July, 2012. (EHU-KZAA-TR:2012-03) |
| Abstract: Methods for generating a new population are a fundamental component of estimation of distribution algorithms (EDAs). They serve to transfer the information contained in the probabilistic model to the new generated population. In EDAs based on Markov networks, methods for generating new populations usually discard information contained in the model to gain in efficiency. Other methods like Gibbs sampling use information about all interactions in the model but are computationally very costly. In this paper we propose new methods for generating new solutions in EDAs based on Markov networks. We introduce approaches based on inference methods for computing the most probable configurations and model-based template recombination. We show that the application of different variants of inference methods can increase the EDAs’ convergence rate and reduce the number of function evaluations needed to find the optimum of binary and non-binary discrete functions. |
BibTeX:
@techreport{Santana_et_al:2012j,
author = {R. Santana and A. Mendiburu and J. A. Lozano},
title = {Using network mesures to test evolved NK-landscapes},
school = {Department of Computer Science and Artificial Intelligence, University of the Basque Country},
year = {2012},
number = {EHU-KZAA-TR:2012-03},
url = {http://hdl.handle.net/10810/9180}
}
|
| Santana R, Armañanzas R, Bielza C and Larrañaga P (2013), "Network measures for information extraction in evolutionary algorithms", International Journal of Computational Intelligence Systems. Vol. 6(6), pp. 1163-1188. |
| Abstract: Problem domain information extraction is a critical issue in many real-world optimization problems. Increasing the repertoire of techniques available in evolutionary algorithms with this purpose is fundamental for extending the applicability of these algorithms. In this paper we introduce a unifying information mining approach for evolutionary algorithms. Our proposal is based on a division of the stages where structural modelling of the variables interactions is applied. Particular topological characteristics induced from different stages of the modelling process are identified. Network theory is used to harvest problem structural information from the learned probabilistic graphical models (PGMs). We show how different statistical measures, previously studied for networks from different domains, can be applied to mine the graphical component of PGMs. We provide evidence that the computed measures can be employed for studying problem difficulty, classifying different problem instances and predicting the algorithm behavior. |
BibTeX:
@article{Santana_et_al:2013,
author = {R. Santana and R. Armañanzas and C. Bielza and P. Larrañaga},
title = {Network measures for information extraction in evolutionary algorithms},
journal = {International Journal of Computational Intelligence Systems},
year = {2013},
volume = {6},
number = {6},
pages = {1163-1188},
url = {https://www.atlantis-press.com/article/25868449.pdf}
}
|
| Santana R, McKay RI and Lozano JA (2013), "Symmetry in evolutionary and estimation of distribution algorithms", In Proceedings of the 2013 Congress on Evolutionary Computation CEC-2013. Cancun, Mexico , pp. 2053-2060. IEEE Press. |
| Abstract: Symmetry has hitherto been studied piecemeal in a variety of evolutionary computation domains, with little consistency between the definitions. Here we provide formal definitions of symmetry that are consistent across the field of evolutionary computation. We propose a number of evolutionary and estimation of distribution algorithms suitable for variable symmetries in Cartesian power domains, and compare their utility, integration of the symmetry knowledge with the probabilistic model of an EDA yielding the best outcomes. We test the robustness of the algorithm to inexact symmetry, finding adequate performance up to about 1% noise. Finally, we present evidence that such symmetries, if not known a priori, may be learnt during evolution. |
BibTeX:
@inproceedings{Santana_et_al:2013a,
author = {R. Santana and R. I. McKay and J. A. Lozano},
title = {Symmetry in evolutionary and estimation of distribution algorithms},
booktitle = {Proceedings of the 2013 Congress on Evolutionary Computation CEC-2013},
publisher = {IEEE Press},
year = {2013},
pages = {2053-2060},
url = {https://ieeexplore.ieee.org/document/6557811}
}
|
| Santana R, Mendiburu A and Lozano JA (2013), "Message passing methods for estimation of distribution algorithms based on Markov networks", In Proceedings of the 4th Conference on Swarm, Evolutionary, and Memetic Computing (SEMCCO-2013). Chennai, India , pp. 419-430. Springer. |
| Abstract: Sampling methods are a fundamental component of estimation of distribution algorithms (EDAs). In this paper we propose new methods for generating solutions in EDAs based on Markov networks. These methods are based on the combination of message passing algorithms with decimation techniques for computing the maximum a posteriori solution of a probabilistic graphical model. The performance of the EDAs on a family of non-binary deceptive functions shows that the introduced approach improves results achieved with the sampling methods traditionally used by EDAs based on Markov networks. |
BibTeX:
@inproceedings{Santana_et_al:2013b,
author = {R. Santana and A. Mendiburu and J. A. Lozano},
title = {Message passing methods for estimation of distribution algorithms based on Markov networks},
booktitle = {Proceedings of the 4th Conference on Swarm, Evolutionary, and Memetic Computing (SEMCCO-2013)},
publisher = {Springer},
year = {2013},
pages = {419-430},
url = {https://link.springer.com/chapter/10.1007/978-3-319-03756-1_38}
}
|
| Santana R, Mendiburu A and Lozano JA (2013), "Critical issues in model-based surrogate functions in estimation of distribution algorithms", In Proceedings of the 4th Conference on Swarm, Evolutionary, and Memetic Computing (SEMCCO-2013). Chennai, India , pp. 1-13. Springer. |
| Abstract: In many optimization domains the solution of the problem can be made more efficient by the construction of a surrogate fitness model. Estimation of distribution algorithms (EDAs) are a class of evolutionary algorithms particularly suitable for the conception of model-based surrogate techniques. Since EDAs generate probabilistic models, it is natural to use these models as surrogates. However, there exist many types of models and methods to learn them. The issues involved in the conception of model-based surrogates for EDAs are various and some of them have received scarce attention in the literature. In this position paper, we propose a unified view for model-based surrogates in EDAs and identify a number of critical issues that should be dealt with in order to advance the research in this area. |
BibTeX:
@inproceedings{Santana_et_al:2013c,
author = {R. Santana and A. Mendiburu and J. A. Lozano},
title = {Critical issues in model-based surrogate functions in estimation of distribution algorithms},
booktitle = {Proceedings of the 4th Conference on Swarm, Evolutionary, and Memetic Computing (SEMCCO-2013)},
publisher = {Springer},
year = {2013},
pages = {1-13},
url = {https://link.springer.com/chapter/10.1007/978-3-319-03756-1_1}
}
|
| Santana R, McGarry L, Bielza C, Larrañaga P and Yuste R (2013), "Classification of neocortical interneurons using affinity propagation", Frontiers in Neural Circuits. Vol. 7, pp. 1-13. |
| Abstract: In spite of over a century of research on cortical circuits, it is still unknown how many classes of cortical neurons exist. In fact, neuronal classification is a difficult problem because it is unclear how to designate a neuronal cell class and what are the best characteristics to define them. Recently, unsupervised classifications using cluster analysis based on morphological, physiological, or molecular characteristics, have provided quantitative and unbiased identification of distinct neuronal subtypes, when applied to selected datasets. However, better and more robust classification methods are needed for increasingly complex and larger datasets. Here, we explored the use of affinity propagation, a recently developed unsupervised classification algorithm imported from machine learning, which gives a representative example or exemplar for each cluster. As a case study, we applied affinity propagation to a test dataset of 337 interneurons belonging to four subtypes, previously identified based on morphological and physiological characteristics. We found that affinity propagation correctly classified most of the neurons in a blind, non-supervised manner. Affinity propagation outperformed Ward's method, a current standard clustering approach, in classifying the neurons into 4 subtypes. Affinity propagation could therefore be used in future studies to validly classify neurons, as a first step to help reverse engineer neural circuits. |
BibTeX:
@article{Santana_et_al:2013d,
author = {R. Santana and L. McGarry and C. Bielza and P. Larrañaga and R. Yuste},
title = {Classification of neocortical interneurons using affinity propagation},
journal = {Frontiers in Neural Circuits},
year = {2013},
volume = {7},
pages = {1-13},
url = {https://www.frontiersin.org/articles/10.3389/fncir.2013.00185/full}
}
|
| Santana R, Bielza C and Larrañaga P (2013), "Changing conduction delays to maximize the number of polychronous groups with an estimation of distribution algorithm". Research Report at: Department of Artificial Intelligence, Faculty of Informatics, Technical University of Madrid. (UPM-FI/DIA/2013-1) |
| Abstract: Spiking network polychronization refers to reproducible time-locked but not synchronous firing patterns with millisecond precision. There are many factors that influence the number of polychronous groups in spiking networks. In this paper we address the question of maximizing the number of polychronous groups by biasing the axonal conduction delay values of the spiking network. An estimation of distribution algorithm (EDA) that learns tree models to search for spiking networks with optimal delay configurations is used. An analysis of the evolved networks show a higher number of polychronous groups with respect to networks with random delay values. We evaluate to what extent the spiking network structure is reflected in the probabilistic models learned by the EDA. Finally, we introduce the informative edge ranking measure r that quantifies how much of the original spiking network structure is captured in the tree models. |
BibTeX:
@techreport{Santana_et_al:2013e,
author = {R. Santana and C. Bielza and P. Larrañaga},
title = {Changing conduction delays to maximize the number of polychronous groups with an estimation of distribution algorithm},
school = {Department of Artificial Intelligence, Faculty of Informatics, Technical University of Madrid},
year = {2013},
number = {UPM-FI/DIA/2013-1},
url = {https://www.researchgate.net/publication/275408208_Changing_conduction_delays_to_maximize_the_number_of_polychronous_groups_with_an_estimation_of_distribution_algorithm}
}
|
| Santana R, Mendiburu A and Lozano JA (2013), "Extending the use of message passing algorithms to problems with unknown structure", In 2013 NIPS Workshop on Bayesian Optimization. , pp. 1-8. |
| Abstract: We investigate the behavior of message passing algorithms (MPAs) on approximate probabilistic graphical models (PGMs) learned in the context of optimization. We use the framework of estimation of distribution algorithms (EDAs), a class of optimization algorithms that learn in each iteration a PGM and sample new solutions from it. The impact that including the most probable configuration of the model has for EDAs is evaluated using a variety of MPAs on different instances of the Ising problem. |
BibTeX:
@inproceedings{Santana_et_al:2013f,
author = {Santana, Roberto and Mendiburu, Alexander and Lozano, Jose A},
title = {Extending the use of message passing algorithms to problems with unknown structure},
booktitle = {2013 NIPS Workshop on Bayesian Optimization},
year = {2013},
pages = {1--8},
url = {https://www.researchgate.net/profile/Roberto-Santana/publication/275408229_Extending_the_use_of_message_passing_algorithms_to_problems_with_unknown_structure/links/553bb6120cf29b5ee4b87c0b/Extending-the-use-of-message-passing-algorithms-to-problems-with-unknown-structure.pdf}
}
|
| Santana R, Mendiburu A and Lozano JA (2015), "Evolving MNK-landscapes with structural constraints", In Proceedings of the IEEE Congress on Evolutionary Computation CEC 2015. Sendai, Japan , pp. 1364-1371. IEEE press. |
| Abstract: In this paper we propose a method for the generation of instances of the MNK-landscapes that maximize different measures used to characterize multi-objective problems. In contrast to previous approaches, the introduced algorithm works by modifying the neighborhood structure of the variables of the MNK-landscape while keeping fixed the local parameters of its functions. A variant of the algorithm is presented to deal with situations in which the exhaustive enumeration of search space is unfeasible. We show how the introduced method can be used to generate instances with an increased number of solutions in the Pareto front. Furthermore, we investigate whether direct optimization of the correlation between objectives can be used as an indirect method to increase the size of the Pareto fronts of the generated instances. |
BibTeX:
@inproceedings{Santana_et_al:2015,
author = {R. Santana and A. Mendiburu and J. A. Lozano},
title = {Evolving MNK-landscapes with structural constraints},
booktitle = {Proceedings of the IEEE Congress on Evolutionary Computation CEC 2015},
publisher = {IEEE press},
year = {2015},
pages = {1364-1371},
url = {https://ieeexplore.ieee.org/document/7257047}
}
|
| Santana R, Mendiburu A and Lozano JA (2015), "Multi-objective NM-landscapes", In Proceedings of the Companion Publication of the 2015 on Genetic and Evolutionary Computation Conference. Madrid, Spain , pp. 1477-1478. |
| Abstract: In this paper we propose an extension of the NM-landscape to model multi-objective problems (MOPs). We illustrate the link between the introduced model and previous landscapes used to study MOPs. Empirical results are presented for a variety of configurations of the multi-objective NM-landscapes. |
BibTeX:
@inproceedings{Santana_et_al:2015b,
author = {R. Santana and A. Mendiburu and J. A. Lozano},
title = {Multi-objective NM-landscapes},
booktitle = {Proceedings of the Companion Publication of the 2015 on Genetic and Evolutionary Computation Conference},
year = {2015},
pages = {1477--1478},
url = {https://dl.acm.org/doi/abs/10.1145/2739482.2764704}
}
|
| Santana R, Mendiburu A and Lozano JA (2015), "Computing factorized approximations of Pareto-fronts using mNM-landscapes and Boltzmann distributions", CoRR. Vol. abs/1512.03466 |
| Abstract: NM-landscapes have been recently introduced as a class of tunable rugged models. They are a subset of the general interaction models where all the interactions are of order less or equal M. The Boltzmann distribution has been extensively applied in single-objective evolutionary algorithms to implement selection and study the theoretical properties of model-building algorithms. In this paper we propose the combination of the multi-objective NM-landscape model and the Boltzmann distribution to obtain Pareto-front approximations. We investigate the joint effect of the parameters of the NM-landscapes and the probabilistic factorizations in the shape of the Pareto front approximations. |
BibTeX:
@article{Santana_et_al:2015c,
author = {Roberto Santana and Alexander Mendiburu and José Antonio Lozano},
title = {Computing factorized approximations of Pareto-fronts using mNM-landscapes and Boltzmann distributions},
journal = {CoRR},
year = {2015},
volume = {abs/1512.03466},
url = {http://arxiv.org/abs/1512.03466}
}
|
| Santana R, Mendiburu A and Lozano JA (2015), "Multi-view classification of psychiatric conditions based on saccades", Applied Soft Computing. Vol. 31, pp. 308-316. Elsevier. |
| Abstract: Early diagnosis of psychiatric conditions can be enhanced by taking into account eye movement behavior. However, the implementation of prediction algorithms which are able to assist physicians in the diagnostic is a difficult task. In this paper we propose, for the first time, an automatic approach for classification of multiple psychiatric conditions based on saccades. In particular, the goal is to classify 6 medical conditions: Alcoholism, Alzheimer's disease, opioid dependence (two groups of subjects with measurements respectively taken prior to and after administering synthetic opioid), Parkinson's disease, and Schizophrenia. Our approach integrates different feature spaces corresponding to complementary characterizations of the saccadic behavior. We define a multi-view model of saccades in which the feature representations capture characteristic temporal and amplitude patterns of saccades. Four of the current most advanced classification methods are used to discriminate among the psychiatric conditions and leave-one-out cross-validation is used to evaluate the classifiers. Classification accuracies well above the chance levels are obtained for the different classification tasks investigated. The confusion matrices reveal that it is possible to separate conditions into different groups. We conclude that using relatively simple descriptors of the saccadic behavior it is possible to simultaneously classify among 6 different types of psychiatric conditions. Conceptually, our multi-view classification method excels other approaches that focus on statistical differences in the saccadic behavior of cases and controls because it can be used for predicting unseen cases. Classification integrating different characterizations of the saccades can actually help to predict the conditions of new patients, opening the possibility to integrate automatic analysis of saccades as a practical procedure for differential diagnosis in Psychiatry. |
BibTeX:
@article{Santana_et_al:2015d,
author = {Santana, Roberto and Mendiburu, Alexander and Lozano, Jose A},
title = {Multi-view classification of psychiatric conditions based on saccades},
journal = {Applied Soft Computing},
publisher = {Elsevier},
year = {2015},
volume = {31},
pages = {308--316},
url = {https://www.sciencedirect.com/science/article/abs/pii/S1568494615001398}
}
|
| Santana R, Zhu Z and Katzgraber HG (2016), "Evolutionary Approaches to Optimization Problems in Chimera Topologies", In Proceedings of the 2016 Conference on Genetic and Evolutionary Computation (GECCO-2016). , pp. 397-404. |
| Abstract: Chimera graphs define the topology of one of the first commercially available quantum computers. A variety of optimization problems have been mapped to this topology to evaluate the behavior of quantum enhanced optimization heuristics in relation to other optimizers, being able to efficiently solve problems classically to use them as benchmarks for quantum machines. In this paper we investigate for the first time the use of Evolutionary Algorithms (EAs) on Ising spin glass instances defined on the Chimera topology. Three genetic algorithms (GAs) and three estimation of distribution algorithms (EDAs) are evaluated over 1000 hard instances of the Ising spin glass constructed from Sidon sets. We focus on determining whether the information about the topology of the graph can be used to improve the results of EAs and on identifying the characteristics of the Ising instances that influence the success rate of GAs and EDAs. |
BibTeX:
@inproceedings{Santana_et_al:2016,
author = {Santana, Roberto and Zhu, Zheng and Katzgraber, Helmut G},
title = {Evolutionary Approaches to Optimization Problems in Chimera Topologies},
booktitle = {Proceedings of the 2016 Conference on Genetic and Evolutionary Computation (GECCO-2016)},
year = {2016},
pages = {397--404},
url = {https://arxiv.org/abs/1608.05105}
}
|
| Santana R, Mendiburu A and Lozano JA (2016), "A review of message passing algorithms in estimation of distribution algorithms", Natural Computing. Vol. 15(1), pp. 165-180. Springer Netherlands. |
| Abstract: Message passing algorithms (MPAs) have been traditionally used as an inference method in probabilistic graphical models. Some MPA variants have recently been introduced in the field of estimation of distribution algorithms (EDAs) as a way to improve the efficiency of these algorithms. Multiple developments on MPAs point to an increasing potential of these methods for their application as part of hybrid EDAs. In this paper we review recent work on EDAs that apply MPAs and propose ways to further extend the useful synergies between MPAs and EDAs. Furthermore, we analyze some of the implications that MPA developments can have in their future application to EDAs and other evolutionary algorithms. |
BibTeX:
@article{Santana_et_al:2016a,
author = {Santana, Roberto and Mendiburu, Alexander and Lozano, Jose A},
title = {A review of message passing algorithms in estimation of distribution algorithms},
journal = {Natural Computing},
publisher = {Springer Netherlands},
year = {2016},
volume = {15},
number = {1},
pages = {165--180},
url = {https://link.springer.com/article/10.1007/s11047-014-9473-2}
}
|
| Santana R, Sirbiladze G, Ghvaberidze B and Matsaberidze B (2017), "A comparison of probabilistic-based optimization approaches for vehicle routing problems", In 2017 IEEE Congress on Evolutionary Computation (CEC). , pp. 2606-2613. |
| Abstract: Estimation of distribution algorithms (EDAs) are evolutionary algorithms that use probabilistic modeling to lead a more efficient search for optimal solutions. While EDAs have been applied to several types of optimization problems, they exhibit some limitations to deal with constrained optimization problems. More study and understanding of how can EDAs deal with these problems is required. In this paper we investigate the application of EDAs to a version of the vehicle routing problem in which solutions should satisfy a number of constraints involving the customers, the fleet vehicle, and the items to be delivered. For this problem, we compare two different representations of the solutions, and apply EDAs that use three probabilistic models with different characteristics. Our results show that the combination of an integer representation with tree-based probabilistic model produces the best results and is able to solve vehicle routing problems that contain over thousands of promising paths. |
BibTeX:
@inproceedings{Santana_et_al:2017,
author = {Santana, Roberto and Sirbiladze, Gia and Ghvaberidze, Bezhan and Matsaberidze, Bidzina},
title = {A comparison of probabilistic-based optimization approaches for vehicle routing problems},
booktitle = {2017 IEEE Congress on Evolutionary Computation (CEC)},
year = {2017},
pages = {2606--2613},
url = {https://ieeexplore.ieee.org/document/7969622}
}
|
| Santana R, Marti L and Zhang M (2019), "GP-based methods for domain adaptation: Using brain decoding across subjects as a test-case", Genetic Programming and Evolvable Machines. Vol. 20(3), pp. 385-411. Springer. |
| Abstract: Research on classifier transferability intends that the information gathered in the solution of a given classification problem could be reused in the solution of similar or related problems. We propose the evolution of transferable classifiers based on the use of multi-objective genetic programming and new fitness-functions that evaluate the amount of transferability. We focus on the domain adaptation scenario in which the problem to be solved is the same in the source and target domains, but the distribution of data is different between domains. As a real-world test case we address the brain decoding problem, whose goal is to predict the stimulus presented to a subject from the analysis of his brain activity. Brain decoding across subjects attempts to reuse the classifiers learned from some subjects in the classification of the others. We evolved GP-based classifiers using different variants of the introduced approach to test their effectiveness on data obtained from a brain decoding experiment involving 16 subjects. Our results show that the GP-based classifiers evolved trying to maximize transferability are able to improve classification accuracy over other classical classifiers that incorporate domain adaptation methods. Moreover, after comparing our algorithm to importance-weighted cross validation (in conjunction with many ML methods), we conclude that our approach achieves state of the art results in terms of transferability. |
BibTeX:
@article{Santana_et_al:2019,
author = {R. Santana and L. Marti and M. Zhang},
title = {GP-based methods for domain adaptation: Using brain decoding across subjects as a test-case},
journal = {Genetic Programming and Evolvable Machines},
publisher = {Springer},
year = {2019},
volume = {20},
number = {3},
pages = {385--411},
url = {https://link.springer.com/article/10.1007/s10710-019-09352-6}
}
|
| Santana T, Moreno J, Petzold G, Santana R and Saez-Trautmann G (2020), "Evaluation of the Temperature and Time in Centrifugation-Assisted Freeze Concentration", Applied Sciences. Vol. 10(24), pp. 9130. MPDI. |
| Abstract: Centrifugation is a technique applied to assist in the freeze concentration of fruit juices and solutions. The aim of this work was to study the influence of the time–temperature parameters on the centrifugation process as a technique applied to assist in the first cycle of the freeze concentration of blueberry juice. A completely randomized 4 × 3 factorial design was performed using temperature and time as the factors, and the response variables included the percentage of concentrate, efficiency and solutes recovered. The results were evaluated using multiple linear regression, random forest regression, and Gaussian processes. The solid content in the concentrate doubled compared to the initial sample (18 °Brix) and approached 60% in the first cycle of blueberry juice freeze concentration. The combination of factors affected the percentage of the concentrate and solutes recovered, and the optimum of concentration was obtained at 15 °C with a centrifugation time of 20 min. Gaussian processes are suggested as suitable machine learning techniques for modelling the quantitative effect of the relevant factors in the centrifugation process. |
BibTeX:
@article{Santana_et_al:2020,
author = {T. Santana and J. Moreno and G. Petzold and R. Santana and G. Saez-Trautmann},
title = {Evaluation of the Temperature and Time in Centrifugation-Assisted Freeze Concentration},
journal = {Applied Sciences},
publisher = {MPDI},
year = {2020},
volume = {10},
number = {24},
pages = {9130},
url = {https://www.mdpi.com/2076-3417/10/24/9130}
}
|
| Santana R, Liefooghe A and Derbel B (2022), "Boomerang-shaped neural embeddings for NK landscapes", In Proceedings of the Genetic and Evolutionary Computation Conference. , pp. 858-866. |
| Abstract: Understanding the landscape underlying NK models is of fundamental interest. Different representations have been proposed to better understand how the ruggedness of the landscape is influenced by the model parameters, such as the problem dimension, the degree of non-linearity and the structure of variable interactions. In this paper, we propose to use neural embedding, that is a continuous vectorial representation obtained as a result of applying a neural network to a prediction task, in order to investigate the characteristics of NK landscapes. The main assumption is that neural embeddings are able to capture important features that reflect the difficulty of the landscape. We propose a method for constructing NK embeddings, together with metrics for evaluating to what extent this embedding space encodes valuable information from the original NK landscape. Furthermore, we study how the embedding dimensionality and the parameters of the NK model influence the characteristics of the NK embedding space. Finally, we evaluate the performance of optimizers that solve the continuous representations of NK models by searching for solutions in the embedding space. |
BibTeX:
@inproceedings{Santana_et_al:2022,
author = {Santana, Roberto and Liefooghe, Arnaud and Derbel, Bilel},
title = {Boomerang-shaped neural embeddings for NK landscapes},
booktitle = {Proceedings of the Genetic and Evolutionary Computation Conference},
year = {2022},
pages = {858--866},
url = {https://dl.acm.org/doi/abs/10.1145/3512290.3528856}
}
|
| Santana R (1996), "Algoritmos Genéticos para un problema de caminos Hamiltonianos". Thesis at: Department of Computer Science, University of Havana., June, 1996.
[BibTeX] |
BibTeX:
@phdthesis{Santana:1996,
author = {R. Santana},
title = {Algoritmos Genéticos para un problema de caminos Hamiltonianos},
school = {Department of Computer Science, University of Havana},
year = {1996},
note = {In Spanish}
}
|
| Santana R (1998), "Using Evolutionary Computing and Artificial Life in Theoretical Ecology: A preliminary approach"
[BibTeX] |
BibTeX:
@unpublished{Santana:1998,
author = {Roberto Santana},
title = {Using Evolutionary Computing and Artificial Life in Theoretical Ecology: A preliminary approach},
year = {1998}
}
|
| Santana R (1999), "Towards an intelligent genetic search: Defining measures of convergence", In Proceedings of the students sessions, ACAI'99. Chania, Greece , pp. 41-42. |
| Abstract: The approach we discuss here is in line with techniques that change or adapt the parameter values as the search progresses, nevertheless it exhibits two main differences with previous work on this topic. First, the analysis has been thought to be applied not only to genetic algorithms (GAs) but also to other Population Based Search Methods that use Selection (PBSMS). Second, adaptation is achieved by considering rules able to change the application of different operators along the search, and not only by adapting the parameters. For reasons of space we concentrate here in the question of defining measures that could allow to a PBSMS to receive a feedback about its own behavior, and use this information in the next step of the algorithm to improve the search. |
BibTeX:
@inproceedings{Santana:1999,
author = {R. Santana},
title = {Towards an intelligent genetic search: Defining measures of convergence},
booktitle = {Proceedings of the students sessions, ACAI'99},
year = {1999},
pages = {41-42},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/1999/[Santana,1999].pdf}
}
|
| Santana R (2002), "An analysis of the performance of the mixture of trees factorized distribution algorithm when priors and adaptive learning are used". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba, March, 2002. (ICIMAF 2002-180) |
| Abstract: This paper analyzes the behavior of the Mixture of Trees Factorized Distribution Algorithm (MT-FDA) when priors are incorporated. It is shown that the addition of priors that relate the rate of mutation like effect during the search. Adaptive priors that relate the rate of mutation to the quality of the search are also introduced. Additionally, the learning step of the MT-FDA is changed to avoid the overfitting fo data. The results of the experiments show that our proposals improve the trade off between exploration and exploitation displayed by the MT-FDA. |
BibTeX:
@techreport{Santana:2002,
author = {Roberto Santana},
title = {An analysis of the performance of the mixture of trees factorized distribution algorithm when priors and adaptive learning are used},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {2002},
number = {ICIMAF 2002-180},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/TechReports/RepMutMTFDA.pdf}
}
|
| Santana R (2002), "Study of neighborhood search operators for unitation functions". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba, March, 2002. (ICIMAF 2002-185) |
| Abstract: In this paper we study the behavior of neighborhood search algorithms in optimization of unitation functions. The in uence of two neighborhood search strategies is analyzed. The expected number of steps required by these algorithms to reach the optimum is derived. The analytical results achieved correspond to previous simulations. |
BibTeX:
@techreport{Santana:2002a,
author = {Roberto Santana},
title = {Study of neighborhood search operators for unitation functions},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {2002},
number = {ICIMAF 2002-185},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/TechReports/ReportNeigMethods4.pdf}
}
|
| Santana R (2003), "MCMC in the approximation and optimization of additive functions: A preliminary study". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba |
| Abstract: This unpublished paper describes the results of different optimization algorithms based on exact and approximate sampling on the solution of additive functions |
BibTeX:
@techreport{Santana:2003,
author = {Roberto Santana},
title = {MCMC in the approximation and optimization of additive functions: A preliminary study},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {2003},
note = {Unpublished manuscript}
}
|
| Santana R (2003), "Factorized Distribution Algorithms: selection without selected population". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba, September, 2003. (ICIMAF 2003-240) |
| Abstract: In this paper we investigate the problem of an efficient implementation of the selection step in Factorized Distribution Algorithms. We demonstrate that while in Genetic Algorithms the selection operator needs the creation of a selected population, in Factorized Distribution Algorithms this is not always the case. |
BibTeX:
@techreport{Santana:2003a,
author = {Roberto Santana},
title = {Factorized Distribution Algorithms: selection without selected population},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {2003},
number = {ICIMAF 2003-240},
url = {http://www.sc.ehu.es/ccwbayes/members/rsantana/papers/TechReports/ReportCECSel.pdf}
}
|
| Santana R (2003), "Estimation of Distribution Algorithms with Kikuchi approximations: Part I". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba, September, 2003. (ICIMAF 2003-242)
[BibTeX] |
BibTeX:
@techreport{Santana:2003b,
author = {Roberto Santana},
title = {Estimation of Distribution Algorithms with Kikuchi approximations: Part I},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {2003},
number = {ICIMAF 2003-242}
}
|
| Santana R (2003), "A Markov network based factorized distribution algorithm for optimization", In Proceedings of the 14th European Conference on Machine Learning (ECML-PKDD 2003). Dubrovnik, Croatia Vol. 2837, pp. 337-348. Springer. |
| Abstract: In this paper we propose a population based optimization method that uses the estimation of probability distributions. To represent an approximate factorization of the probability, the algorithm employs a junction graph constructed from an independence graph. We show that the algorithm extends the representation capabilities of previous algorithms that use factorizations. A number of functions are used to evaluate the performance of our proposal. The results of the experiments show that the algorithm is able to optimize the functions, outperforming other evolutionary algorithms that use factorizations. |
BibTeX:
@inproceedings{Santana:2003c,
author = {R. Santana},
title = {A Markov network based factorized distribution algorithm for optimization},
booktitle = {Proceedings of the 14th European Conference on Machine Learning (ECML-PKDD 2003)},
publisher = {Springer},
year = {2003},
volume = {2837},
pages = {337-348},
url = {http://dx.doi.org/10.1007/b13633}
}
|
| Santana R (2003), "Estimation of Distribution Algorithms with Kikuchi approximations: Part II". Research Report at: Institute of Cybernetics, Mathematics and Physics. Havana, Cuba, September, 2003. (ICIMAF 2003)
[BibTeX] |
BibTeX:
@techreport{Santana:2003d,
author = {Roberto Santana},
title = {Estimation of Distribution Algorithms with Kikuchi approximations: Part II},
school = {Institute of Cybernetics, Mathematics and Physics},
year = {2003},
number = {ICIMAF 2003}
}
|
| Santana R (2003), "Study of neighborhood search operators for unitation functions", In Proceedings of the 17th European Simulation Multiconference ESM-2003. Nottingham, England , pp. 272-277. |
| Abstract: In this paper we study the behavior of neighborhood search algorithms in optimization of unitation functions. The influence of two neighborhood search strategies is analyzed. The expected number of steps required by these algorithms to reach the optimum is derived. The analytical results achieved correspond to previous simulations. |
BibTeX:
@inproceedings{Santana:2003e,
author = {R. Santana},
title = {Study of neighborhood search operators for unitation functions},
booktitle = {Proceedings of the 17th European Simulation Multiconference ESM-2003},
year = {2003},
pages = {272--277},
url = {www.scs-europe.net/services/esm2003/PDF/AS-21.pdf}
}
|
| Santana R (2003), "Factorized Distribution Algorithms: Selection without selected population", In Proceedings of the 17th European Simulation Multiconference ESM-2003. Nottingham, England , pp. 91-97. |
| Abstract: In this paper we investigate the problem of an efficient implementation of the selection step in Factorized Distribution Algorithms. We demonstrate that while in Genetic Algorithms the selection operator needs the creation of a selected population, in Factorized Distribution Algorithms this is not always the case. |
BibTeX:
@inproceedings{Santana:2003f,
author = {R. Santana},
title = {Factorized Distribution Algorithms: Selection without selected population},
booktitle = {Proceedings of the 17th European Simulation Multiconference ESM-2003},
year = {2003},
pages = {91--97},
url = {https://www.scs-europe.net/services/esm2003/PDF/AI-09.pdf}
}
|
| Santana R (2003), "Automatic construction of valid region based decompositions of graphs", In Proceedings of the Conference of Cuban Society of Computer Science and Mathematics COMPUMAT-2003. Sancti Spiritus, Cuba |
| Abstract: In this paper we propose an algorithm for the construction of valid region based decompositions of graphs. The algorithm is inspired in the Cluster Variation Method used in Statisitical Physics. We illustrate our approach using a number of examples, and show that the algorithm is able to deal with cases where other algorithms fail. |
BibTeX:
@inproceedings{Santana:2003g,
author = {R. Santana},
title = {Automatic construction of valid region based decompositions of graphs},
booktitle = {Proceedings of the Conference of Cuban Society of Computer Science and Mathematics COMPUMAT-2003},
year = {2003}
}
|
| Santana R (2003), "Exact Gibbs sampling in optimization", In Proceedings of the Conference of Cuban Society of Computer Science and Mathematics COMPUMAT-2003. Sancti Spiritus, Cuba |
| Abstract: In this paper we evaluate the convenience of using Exact sampling methods based on Markov chains like optimization algorithms |
BibTeX:
@inproceedings{Santana:2003h,
author = {R. Santana},
title = {Exact Gibbs sampling in optimization},
booktitle = {Proceedings of the Conference of Cuban Society of Computer Science and Mathematics COMPUMAT-2003},
year = {2003}
}
|
| Santana R (2004), "Modelación probabilística basada en modelos gráficos no dirigidos en Algoritmos Evolutivos con Estimación de Distribuciones". Research Report at: Instituto de Cibernética, Matemática y Física. |
| Abstract: Esta tesis trata sobre el proceso de modelación probabilística en una clase de algoritmos deoptimización basados en poblaciones llamados Algoritmos Evolutivos con Estimación de Distribuciones(EDAs). El objetivo principal de la tesis es el desarrollo de EDAs con modelos probabilísticos complejos, basados en modelos gráficos no dirigidos, y capaces de optimizar funciones que no pueden ser optimizadas por EDAs de modelos probabilísticos basados en estructuras gráficas simplemente conectadas. El problema es abordado a partir de dos enfoques diferentes. El primer enfoque considera las mezclas de distribuciones como modelo probabilístico. Se aborda el aprendizaje y muestreo de mezclas con vistas a su inserción en el marco de los EDAs. Se propone un método para el aprendizaje de mezclas y modifiaciones a un algoritmo de aprendizaje existente, para el tratamiento del problema de sobreajuste de los datos. Se analizan algunas de las propiedades de las mezclas para la representación de distribuciones. La tesis introduce dos EDAs basados en mezclas. El segundo enfoque busca extender la clase de factorizaciones basadas en modelos gráficos no dirigidos para su aplicación en EDAs. Se introducen los grafos de cliques y la aproximación Kikuchi, y se investiga su capacidad para la representación de dependencias probabilísticas. A partir de estos modelos se introducen dos nuevos EDAs cuyo comportamiento es evaluado en la optimización de diferentes funciones. |
BibTeX:
@phdthesis{Santana:2004a,
author = {R. Santana},
title = {Modelación probabilística basada en modelos gráficos no dirigidos en Algoritmos Evolutivos con Estimación de Distribuciones},
school = {Instituto de Cibernética, Matemática y Física},
year = {2004}
}
|
| Santana R (2003), "Probabilistic modeling based on undirected graphs in Estimation Distribution Algorithms". Thesis at: Institute of Cybernetics, Mathematics, and Physics (ICIMAF). |
| Abstract: This document contains an initial version of the Ph. D. thesis: Modelación probabilística basada en modelos gráficos no dirigidos en Algoritmos Evolutivos con Estimación de Distribuciones written and presented in Spanish in the University of Havana, Cuba. This version contains results that were not included in the Spanish version and some of which were not published later. In particular, the analysis of quadratic measures for junction trees was not been submmitted for peer-review. I decided to publish this document since it contains a good review of initial work on EDAs and it has been already cited by other authors with no knowledge to access to the final Ph.D. in Spanish. |
BibTeX:
@phdthesis{Santana:2004b,
author = {R. Santana},
title = {Probabilistic modeling based on undirected graphs in Estimation Distribution Algorithms},
school = {Institute of Cybernetics, Mathematics, and Physics (ICIMAF)},
year = {2003},
note = {Unpublished document}
}
|
| Santana R (2005), "Estimation of distribution algorithms with Kikuchi approximations", Evolutionary Computation. Vol. 13(1), pp. 67-97. |
| Abstract: The question of finding feasible ways for estimating probability distributions is one of the main challenges for Estimation of Distribution Algorithms (EDAs). To estimate the distribution of the selected solutions, EDAs use factorizations constructed according to graphical models. The class of factorizations that can be obtained from these probability models is highly constrained. Expanding the class of factorizations that could be employed for probability approximation is a necessary step for the conception of more robust EDAs. In this paper we introduce a method for learning a more general class of probability factorizations. The method combines a reformulation of a probability approximation procedure known in statistical physics as the Kikuchi approximation of energy, with a novel approach for finding graph decompositions. We present the Markov Network Estimation of Distribution Algorithm (MN-EDA), an EDA that uses Kikuchi approximations to estimate the distribution, and Gibbs Sampling (GS) to generate new points. A systematic empirical evaluation of MN-EDA is done in comparison with different Bayesian network based EDAs. From our experiments we conclude that the algorithm can outperform other EDAs that use traditional methods of probability approximation in the optimization of functions with strong interactions among their variables. |
BibTeX:
@article{Santana:2005,
author = {R. Santana},
title = {Estimation of distribution algorithms with Kikuchi approximations},
journal = {Evolutionary Computation},
year = {2005},
volume = {13},
number = {1},
pages = {67--97},
url = {http://www.mitpressjournals.org/doi/abs/10.1162/1063656053583496}
}
|
| Santana R (2006), "Advances in Probabilistic Graphical Models for Optimization and Learning. Applications in Protein Modelling". Thesis at: University of the Basque Country. |
| Abstract: The first part focuses on the analysis of undirected graphical models. The ways in which inference and sampling methods based on undirected graphical models can be applied to optimization are discussed. The second part of the thesis introduces a number of properties of the class of Kikuchi approximation that use clique-based decompositions. An algorithm that learns this approximation from data is introduced and evaluated in different types of approximation problem. Region-based decompositions in optimization methods that use inference techniques are analyzed. The second part comprises three chapters. The third part of the thesis addresses the application of optimization algorithms based on graphical models to problems from computational biology. It starts by reviewing a number of computational protein problems. Different proposals that allow the efficient solution of some of these problems are introduced. A link to the results achieved in the first part of the thesis is presented by showing how the probabilistic models and techniques introduced can be added to obtain solutions good enough for these problems. Part four consists of only one chapter that presents the conclusions of the thesis. |
BibTeX:
@phdthesis{Santana:2006,
author = {R. Santana},
title = {Advances in Probabilistic Graphical Models for Optimization and Learning. Applications in Protein Modelling},
school = {University of the Basque Country},
year = {2006}
}
|
| Santana R (2011), "Estimation of distribution algorithms: from available implementations to potential developments", In Companion proceedings of the 2011 Genetic and Evolutionary Computation Conference GECCO-2011. Dublin, Ireland , pp. 679-686. |
| Abstract: This paper focuses on the analysis of estimation of distribution algorithms (EDAs) software. The important role played by EDAs implementations in the usability and range of applications of these algorithms is considered. A survey of available EDA software is presented, and classifications based on the class of programming languages and design strategies used for their implementations are discussed. The paper also reviews different directions to improve current EDA implementations. A number of lines for further expanding the areas of application for EDAs software are proposed. |
BibTeX:
@inproceedings{Santana:2011,
author = {R. Santana},
title = {Estimation of distribution algorithms: from available implementations to potential developments},
booktitle = {Companion proceedings of the 2011 Genetic and Evolutionary Computation Conference GECCO-2011},
year = {2011},
pages = {679-686},
url = {http://dl.acm.org/citation.cfm?id=2002067}
}
|
| Santana R (2012), "MN-EDA and the Use of Clique-Based Factorisations in EDAs", In Markov Networks in Evolutionary Computation. , pp. 73-87. Springer. |
| Abstract: This chapter discusses the important role played by factorisations in the study of EDAs and presents the Markov network estimation of distribution algorithm (MN-EDA) as a classical example of the EDAs based on the use of undirected graphs. The chapter also reviews recent work on the use of clique-based decompositions and other approximations methods inspired in the field of statistical physics with direct application to EDAs. |
BibTeX:
@incollection{Santana:2012,
author = {R. Santana},
editor = {S. Shakya and R. Santana},
title = {MN-EDA and the Use of Clique-Based Factorisations in EDAs},
booktitle = {Markov Networks in Evolutionary Computation},
publisher = {Springer},
year = {2012},
pages = {73-87},
url = {http://dx.doi.org/10.1007/978-3-642-28900-2_5}
}
|
| Santana R (2013), "Multi-objective optimization approach to detecting extremal patterns in social networks", In 2013 Third World Congress on Information and Communication Technologies (WICT 2013). , pp. 196-201. |
| Abstract: This paper introduces the use of extremal patterns as a way to characterize social networks. The concepts of Pareto-dominance, multi-objective optimization, and estimation of distribution algorithms are integrated in a general strategy to compute the multiple extremal patterns. The algorithm is applied to the identification of sets of subjects that have the broadest direct network reachability in a social network extracted from the Reality mining dataset. |
BibTeX:
@inproceedings{Santana:2013,
author = {R. Santana},
title = {Multi-objective optimization approach to detecting extremal patterns in social networks},
booktitle = {2013 Third World Congress on Information and Communication Technologies (WICT 2013)},
year = {2013},
pages = {196--201},
url = {https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7113134}
}
|
| Santana R (2013), "A detailed investigation of classification methods for vowel speech imagery recognition". Research Report at: Department of Computer Science and Artificial Intelligence, University of the Basque Country., December, 2013. (hdl.handle.net/10810/4562) |
| Abstract: Accurate and fast decoding of speech imagery from electroencephalographic (EEG) data could serve as a basis for a new generation of brain computer interfaces (BCIs), more portable and easier to use. However, decoding of speech imagery from EEG is a hard problem due to many factors. In this paper we focus on the analysis of the classification step of speech imagery decoding for a three-class vowel speech imagery recognition problem. We empirically show that different classification subtasks may require different classifiers for accurately decoding and obtain a classification accuracy that improves the best results previously published. We further investigate the relationship between the classifiers and different sets of features selected by the common spatial patterns method. Our results indicate that further improvement on BCIs based on speech imagery could be achieved by carefully selecting an appropriate combination of classifiers for the subtasks involved. |
BibTeX:
@techreport{Santana:2013a,
author = {R. Santana},
title = {A detailed investigation of classification methods for vowel speech imagery recognition},
school = {Department of Computer Science and Artificial Intelligence, University of the Basque Country},
year = {2013},
number = {hdl.handle.net/10810/4562},
url = {https://addi.ehu.es/bitstream/handle/10810/11154/tr13-2.pdf?sequence=1}
}
|
| Santana R (2015), "Supervised classification of vowel speech imagery", In Actas de la XVI Conferencia CAEPIA. , pp. 951-961. |
| Abstract: Accurate and fast decoding of speech imagery from electroencephalographic (EEG) data could serve as a basis for a new generation of brain computer interfaces (BCIs), more portable and easier to use. However, decoding of speech imagery from EEG is a hard problem due to many factors. In this paper we focus on the analysis of the classification step of speech imagery decoding for a three-class vowel speech imagery recognition problem. We empirically show that different classification subtasks may require different classifiers for accurately decoding and obtain a classification accuracy that improves the best results previously published. We further investigate the relationship between the classifiers and different sets of features selected by the common spatial patterns method. Our results indicate that further improvement on BCIs based on speech imagery could be achieved by carefully selecting an appropriate combination of classifiers for the subtasks involved. |
BibTeX:
@inproceedings{Santana:2015,
author = {Santana, Roberto},
title = {Supervised classification of vowel speech imagery},
booktitle = {Actas de la XVI Conferencia CAEPIA},
year = {2015},
pages = {951--961},
url = {https://addi.ehu.es/bitstream/handle/10810/11154/tr13-2.pdf?sequence=1&isAllowed=y}
}
|
| Santana R (2017), "Reproducing and learning new algebraic operations on word embeddings using genetic programming", CoRR. Vol. abs/1702.05624 |
| Abstract: Metaheuristics that explore the decision variables space to construct probabilistic modeling from promising solutions, like estimation of distribution algorithms (EDAs), are becoming very popular in the context of Multi-objective Evolutionary Algorithms (MOEAs). The probabilistic model used in EDAs captures certain statistics of problem variables and their interdependencies. Moreover, the incorporation of local search methods tends to achieve synergy of MOEAs' operators and local heuristics aiming to improve the performance. In this work, we aim to scrutinize the probabilistic graphic model (PGM) presented in Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm (HMOBEDA), which is based on a Bayesian network. Different from traditional EDA-based approaches, the PGM of HMOBEDA provides the joint probability of decision variables, objectives, and configuration parameters of an embedded local search. HMOBEDA has shown to be very competitive on instances of Multi-Objective Knapsack Problem (MOKP), outperforming state-of-the-art approaches. Two variants of HMOBEDA are proposed in this paper using different sample methods. We aim to compare the learnt structure in terms of the probabilistic Pareto Front approximation produced at the end of evolution. Results on instances of MOKP with 2 to 8 objectives show that both proposed variants outperformthe original approach, providing not only the best values for hypervolume and inverted generational distance indicators, butalso a higher diversity in the solution set. |
BibTeX:
@article{Santana:2017,
author = {Roberto Santana},
title = {Reproducing and learning new algebraic operations on word embeddings using genetic programming},
journal = {CoRR},
year = {2017},
volume = {abs/1702.05624},
url = {https://arxiv.org/abs/1702.05624v}
}
|
| Santana R (2017), "Gray-box optimization and factorized distribution algorithms: where two worlds collide", CoRR. Vol. abs/1707.03093 |
| Abstract: The concept of gray-box optimization, in juxtaposition to black-box optimization, revolves about the idea of exploiting the problem structure to implement more efficient evolutionary algorithms (EAs). Work on factorized distribution algorithms (FDAs), whose factorizations are directly derived from the problem structure, has also contributed to show how exploiting the problem structure produces important gains in the efficiency of EAs. In this paper we analyze the general question of using problem structure in EAs focusing on confronting work done in gray-box optimization with related research accomplished in FDAs. This contrasted analysis helps us to identify, in current studies on the use problem structure in EAs, two distinct analytical characterizations of how these algorithms work. Moreover, we claim that these two characterizations collide and compete at the time of providing a coherent framework to investigate this type of algorithms. To illustrate this claim, we present a contrasted analysis of formalisms, questions, and results produced in FDAs and gray-box optimization. Common underlying principles in the two approaches, which are usually overlooked, are identified and discussed. Besides, an extensive review of previous research related to different uses of the problem structure in EAs is presented. The paper also elaborates on some of the questions that arise when extending the use of problem structure in EAs, such as the question of evolvability, high cardinality of the variables and large definition sets, constrained and multi-objective problems, etc. Finally, emergent approaches that exploit neural models to capture the problem structure are covered. |
BibTeX:
@article{Santana:2017a,
author = {Roberto Santana},
title = {Gray-box optimization and factorized distribution algorithms: where two worlds collide},
journal = {CoRR},
year = {2017},
volume = {abs/1707.03093},
url = {https://arxiv.org/abs/1707.03093}
}
|
| Santana R (2021), "Semantic Composition of Word-Embeddings with Genetic Programming", In Heuristics for Optimization and Learning. , pp. 409-423. Springer, Cham.. |
| Abstract: Word-embeddings are vectorized numerical representations of words increasingly applied in natural language processing. Spaces that comprise the embedding representations can capture semantic and other relationships between the words. In this paper we show that it is possible to learn methods for word composition in semantic spaces using genetic programming (GP). We propose to address the creation of word embeddings that have a target semantic content as an automatic program generation problem. We solve this problem using GP. Using a word analogy task as benchmark, we also show that GP-generated programs are able to obtain accuracy values above those produced by the commonly used human-designed rule for algebraic manipulation of word vectors. Finally, we show the robustness of our approach by executing the evolved programs on the word2vec GoogleNews vectors, learned over 3 billion running words, and assessing their accuracy in the same word analogy task. |
BibTeX:
@incollection{Santana:2021,
author = {Santana, R},
title = {Semantic Composition of Word-Embeddings with Genetic Programming},
booktitle = {Heuristics for Optimization and Learning},
publisher = {Springer, Cham.},
year = {2021},
pages = {409--423},
url = {https://link.springer.com/chapter/10.1007/978-3-030-58930-1_27}
}
|
| Santana R (2022), "An embedding space for SARS-CoV-2 epitope-based vaccines", In European Journal of Clinical Investigation. Vol. 52 |
| Abstract: Epitopes are relatively small peptide chains which play an important role in the immune response. They are identified by T cells and B cells which participate in the immune response in humans. Epitope-based vaccines synthesize epitopes that when inoculated stimulate the natural response to a pathogen. This type of vaccines have been proposed for pathogens such as influenza, tuberculosis, dengue, and more recently SARS-CoV-2. In this talk we address the problem of learning an embedding representation of epitopes useful for the design of epitope vaccines. Different criteria should be taken into account in the design of an epitope vaccine, including the biding affinity of the epitopes to one or more major histocompatibility complex (MHC) alleles, the extent to which they cover haplotype distribution of the target population, etc. The goal of the epitope embedding design is capturing in the representation specific immunogenic characteristics of the epitopes in relation to the different MHC alleles. Starting from an original set of peptides (peptide vocabulary), e.g., those extracted from the genome of a pathogen, we propose methods to generate artificial sequences of such peptides. The sequence generation method is used to create large datasets of sequences (vaccine corpora) which are assumed to exhibit some latent semantics related to the way epitopes interact among them and with alleles to provide an immunogenic effect. We use the corpora to create neural epitope embeddings learned in an unsupervised way. We them explore the space of embeddings and discuss how to use them to define intrinsic and extrinsic tasks related to vaccine design. Using a large set of SARS-CoV-2 T cell epitope candidates, we show how to address the vaccine design problem in the epitope embedding space, and provide evidence that such embeddings can be used for solving downstream tasks related to epitope-based vaccine design. |
BibTeX:
@inproceedings{Santana:2022a,
author = {Santana, R},
title = {An embedding space for SARS-CoV-2 epitope-based vaccines},
booktitle = {European Journal of Clinical Investigation},
year = {2022},
volume = {52}
}
|
| Shakya S and Santana R (2008), "An EDA based on local Markov property and Gibbs sampling", In Proceedings of the 2008 Genetic and evolutionary computation conference (GECCO). New York, NY, USA , pp. 475-476. ACM. |
| Abstract: The key ideas behind most of the recently proposed Markov networks based EDAs were to factorise the joint probability distribution in terms of the cliques in the undirected graph. As such, they made use of the global Markov property of the Markov network. Here we presents a Markov Network based EDA that exploits Gibbs sampling to sample from the Local Markov property, the Markovianity, and does not directly model the joint distribution. We call it Markovianity based Optimisation Algorithm. Some initial results on the performance of the proposed algorithm shows that it compares well with other Bayesian network based EDAs |
BibTeX:
@inproceedings{Shakya_and_Santana:2008,
author = {Siddhartha Shakya and R. Santana},
editor = {Maarten Keijzer},
title = {An EDA based on local Markov property and Gibbs sampling},
booktitle = {Proceedings of the 2008 Genetic and evolutionary computation conference (GECCO)},
publisher = {ACM},
year = {2008},
pages = {475--476},
url = {http://dl.acm.org/citation.cfm?id=1389185}
}
|
| Shakya S and Santana R (2012), "A Review of Estimation of Distribution Algorithms and Markov Networks", In Markov Networks in Evolutionary Computation. , pp. 21-37. Springer. |
| Abstract: This chapter reviews some of the popular EDAs based on Markov Networks. It starts by giving introduction to general EDAs and describes the motivation behind their emergence. It then categorises EDAs according to the type of probabilistic models they use (directed model based, undirected model based and common model based) and briefly lists some of the popular EDAs in each categories. It then further focuses on undirected model based EDAs, describes their general workflow and the history, and briefly reviews some of the popular EDAs based on undirected models. It also outlines some of the current research work in this area. |
BibTeX:
@incollection{Shakya_and_Santana:2012a,
author = {S. Shakya and R. Santana},
editor = {S. Shakya and R. Santana},
title = {A Review of Estimation of Distribution Algorithms and Markov Networks},
booktitle = {Markov Networks in Evolutionary Computation},
publisher = {Springer},
year = {2012},
pages = {21-37},
url = {http://dx.doi.org/10.1007/978-3-642-28900-2_2}
}
|
| Shakya S and Santana R (2012), "MOA - Markovian Optimisation Algorithm", In Markov Networks in Evolutionary Computation. , pp. 39-53. Springer. |
| Abstract: In this chapter we describe Markovian Optimisation Algorithm (MOA), one of the recent developments in MN based EDA. It uses the local Markov property to model the dependency and directly sample from it without needing to approximate a complex join probability distribution model. MOA has a much simpler workflow in comparison to its global property based counter parts, since expensive processes to finding cliques, and building and estimating clique potential functions are avoided. The chapter is intended as an introductory chapter, and describes the motivation and the workflow of MOA. It also reviews some of the results obtained with it. |
BibTeX:
@incollection{Shakya_and_Santana:2012b,
author = {S. Shakya and R. Santana},
editor = {S. Shakya and R. Santana},
title = {MOA - Markovian Optimisation Algorithm},
booktitle = {Markov Networks in Evolutionary Computation},
publisher = {Springer},
year = {2012},
pages = {39-53},
url = {http://dx.doi.org/10.1007/978-3-642-28900-2_3}
}
|
| Shakya S, Santana R and Lozano JA (2012), "A Markovianity based optimisation algorithm", Genetic Programming and Evolvable Machines. Vol. 13(2), pp. 159-195. |
| Abstract: Several Estimation of Distribution Algorithms (EDAs) based on Markov networks have been recently proposed. The key idea behind these EDAs was to factorise the joint probability distribution of solution variables in terms of cliques in the undirected graph. As such, they made use of the global Markov property of the Markov network in one form or another. This paper presents a Markov Network based EDA that is based on the use of the local Markov property, the Markovianity, and does not directly model the joint distribution. We call it Markovianity based Optimisation Algorithm. The algorithm combines a novel method for extracting the neighbourhood structure from the mutual information between the variables, with a Gibbs sampler method to generate new points. We present an extensive empirical validation of the algorithm on problems with complex interactions, comparing its performance with other EDAs that use higher order interactions. We extend the analysis to other functions with discrete representation, where EDA results are scarce, comparing the algorithm with state of the art EDAs that use marginal product factorisations. |
BibTeX:
@article{Shakya_et_al:2011,
author = {S. Shakya and R. Santana and J. A. Lozano},
title = {A Markovianity based optimisation algorithm},
journal = {Genetic Programming and Evolvable Machines},
year = {2012},
volume = {13},
number = {2},
pages = {159--195},
url = {http://dx.doi.org/10.1007/s10710-011-9149-y}
}
|
| Sirbiladze G, Khutsishvili I, Sikharulidze A, Manjapharashvili T and Santana R (2020), "A new hesitant fuzzy TOPSIS approach in multi-attribute group decision making", Bulletin of the Georgian National Academy of Sciences. Vol. 14(3), pp. 17-22. |
| Abstract: The proposed multi-attribute decision making methodology applies the TOPSIS (Technique for Order Performance by Similarity to Ideal Solution) approach under hesitant fuzzy environment. The case when the information on the attributes weights is completely unknown is considered. The identification of the weights of attributes which is based on De Luca-Termini information entropy is presented in the context of hesitant fuzzy sets. The ranking of alternatives is performed in accordance with the proximity of their distances to the both fuzzy positive ideal solution (FPIS) and fuzzy negative ideal solution (FNIS). |
BibTeX:
@article{Sirbiladze_et_al:2020,
author = {G. Sirbiladze and I. Khutsishvili and A. Sikharulidze and T. Manjapharashvili and R. Santana},
title = {A new hesitant fuzzy TOPSIS approach in multi-attribute group decision making},
journal = {Bulletin of the Georgian National Academy of Sciences},
year = {2020},
volume = {14},
number = {3},
pages = {17--22},
url = {http://science.org.ge/bnas/vol-14-3.html}
}
|
| Soto MR, Ochoa A and Santana R (2001), "On the use of Polytrees in Evolutionary Computation", Investigación Operacional. Vol. 22(3), pp. 132-138. |
| Abstract: Bayesian networks, are usefull tools for the representation of non-linear interactions among variables. Recently, they have been combined with evolutionary methods to form a new class of optimization algorithms: the Factorized Distribution Algorithms (FDAs). FDAs have been proved to be significantly better than their genetic ancestors. They learn and sample distributions instead of using crossover and mutation operators. Most of the members of the FDAs that have been designed, learn general Bayesian networks. However, in this work we study a FDA that learns polytrees, which are single connected directed graphs. Key words: Bayesian networks, evolutionary algorithms, FDAs. MSC: 90B15, 90C35. RESUMEN Las redes Bayesianas son herramientas muy útiles para la representación de las interacciones no lineales entre las variables. Recientemente estas han sido combinadas con los métodos evolutivos para formar una nueva clase de algoritmos de optimización: los Algoritmos con Distribución Factorizada (FDA). Se ha probado que los FDA son mejores que sus antecesores, los Algoritmos Genéticos, en problemas donde existe una fuerte interacción entre las variables. Estos algoritmos aprenden y simulan distribuciones probabilísticas, en lugar de usar operadores de cruzamiento y mutación. La mayoría de los FDA diseñados hasta el momento aprenden redes Bayesianas generales. Sin embargo, en este trabajo nosotros estudiamos un FDA que aprende poliárboles, los cuales son grafos dirigidos simplemente conectados. Palabras clave: redes Bayesianas, algoritmos evolutivos, FDA. |
BibTeX:
@article{Soto_et_al_2001,
author = {Marta Rosa Soto and Alberto Ochoa and Roberto Santana},
title = {On the use of Polytrees in Evolutionary Computation},
journal = {Investigación Operacional},
year = {2001},
volume = {22},
number = {3},
pages = {132--138},
url = {https://www.researchgate.net/publication/268320573_On_the_use_of_polytrees_in_evolutionary_optimization}
}
|
| Soto MR, Ochoa A, Madera J and Santana R (1999), "Estructuras gráficas simples para algoritmos evolutivos", In Proceedings of the Segundo Encuentro Nacional de Computacionón ENC-99. Pachuca, México , pp. 79-84.
[BibTeX] |
BibTeX:
@inproceedings{Soto_et_al:1999a,
author = {M. R. Soto and A. Ochoa and J. Madera and R. Santana},
title = {Estructuras gráficas simples para algoritmos evolutivos},
booktitle = {Proceedings of the Segundo Encuentro Nacional de Computacionón ENC-99},
year = {1999},
pages = {79--84}
}
|
| Soto D, Sheikh UA, Mei N and Santana R (2020), "Decoding and encoding models reveal the role of mental simulation in the brain representation of meaning", Royal Society open science. Vol. 7(5), pp. 192043. The Royal Society. |
| Abstract: How the brain representation of conceptual knowledge varies as a function of processing goals, strategies and task-factors remains a key unresolved question in cognitive neuroscience. In the present functional magnetic resonance imaging study, participants were presented with visual words during functional magnetic resonance imaging (fMRI). During shallow processing, participants had to read the items. During deep processing, they had to mentally simulate the features associated with the words. Multivariate classification, informational connectivity and encoding models were used to reveal how the depth of processing determines the brain representation of word meaning. Decoding accuracy in putative substrates of the semantic network was enhanced when the depth processing was high, and the brain representations were more generalizable in semantic space relative to shallow processing contexts. This pattern was observed even in association areas in inferior frontal and parietal cortex. Deep information processing during mental simulation also increased the informational connectivity within key substrates of the semantic network. To further examine the properties of the words encoded in brain activity, we compared computer vision models—associated with the image referents of the words—and word embedding. Computer vision models explained more variance of the brain responses across multiple areas of the semantic network. These results indicate that the brain representation of word meaning is highly malleable by the depth of processing imposed by the task, relies on access to visual representations and is highly distributed, including prefrontal areas previously implicated in semantic control. |
BibTeX:
@article{Soto_et_al:2020,
author = {Soto, David and Sheikh, Usman Ayub and Mei, Ning and Santana, Roberto},
title = {Decoding and encoding models reveal the role of mental simulation in the brain representation of meaning},
journal = {Royal Society open science},
publisher = {The Royal Society},
year = {2020},
volume = {7},
number = {5},
pages = {192043},
url = {https://royalsocietypublishing.org/doi/full/10.1098/rsos.192043}
}
|
| Strickler A, Castro O, Pozo A and Santana R (2016), "Investigating selection strategies in multi-objective probabilistic model based algorithms", In 2016 5th Brazilian Conference on Intelligent Systems (BRACIS). Recife, Brazil , pp. 7-12. |
| Abstract: Recent advances on multi-objective evolutionary algorithm (MOEAs) have acknowledged the important role played by selection, replacement, and archiving strategies in the behavior of these algorithms. However, the influence of these methods has been scarcely investigated for the particular class of MOEAs that use probabilistic modeling of the solutions. In this paper we fill this void by proposing an analysis of the role of the aforementioned strategies on an extensive set of bi-objective functions. We focus on the class of algorithms that use Gaussian univariate marginal models, and study how typical selection and replacement strategies used together with this probabilistic model impact the behavior of the search. Our analysis is particularized for a set of bi-objective functions that exhibit a representative set of characteristics (e.g. decomposable, ill-conditioned, non-linear, etc.). The experimental results shows that MOEAs that use simple probabilistic modeling outperform traditional MOEAs based on crossover operators. |
BibTeX:
@inproceedings{Strickler_et_al:2016,
author = {Strickler, Andrei and Castro, Olacir and Pozo, Aurora and Santana, Roberto},
title = {Investigating selection strategies in multi-objective probabilistic model based algorithms},
booktitle = {2016 5th Brazilian Conference on Intelligent Systems (BRACIS)},
year = {2016},
pages = {7--12},
url = {https://ieeexplore.ieee.org/document/7839554}
}
|
| Strickler A, Castro Jr O, Pozo A and Santana R (2018), "An investigation of the selection strategies impact on MOEDAs: CMA-ES and UMDA", Applied Soft Computing. Vol. 62, pp. 963-973. Elsevier. |
| Abstract: Researchers have acknowledged the importance of the selection, replacement, and archiving strategies in the behavior of evolutionary algorithms (EAs). However, these important relationships have not been deeply investigated in terms of Estimation of Distribution algorithms (EDAs). In a preliminary research, we focused on UMDA, a simple EDA that uses univariate Gaussian factorizations. Experimental results confirm that the choice of the selection method can provoke probabilistic modeling to be more effective for some classes of functions. Motivated by these results, this study is extended by evaluating several variants of selection strategies and probabilistic modeling approaches. The aim is to detect possible interactions between these two important components of evolutionary algorithms. Specifically, we use the selection strategies as defined for NSGA2, SPEA2, and IBEA algorithms, and the probabilistic models implemented as part of UMDA and CMA-ES, and a simple crossover operator (SBX). The recently introduced COCO framework comprising 55 bi-objective functions is used as the benchmark for the analysis. The results show that probabilistic modeling has an advantage over the classical genetic operator regardless of the selection method applied. Nevertheless, the results also show that some selection methods have a better performance when applied together with EDAs. |
BibTeX:
@article{Strickler_et_al:2018,
author = {Strickler, Andrei and Castro Jr, Olacir and Pozo, Aurora and Santana, Roberto},
title = {An investigation of the selection strategies impact on MOEDAs: CMA-ES and UMDA},
journal = {Applied Soft Computing},
publisher = {Elsevier},
year = {2018},
volume = {62},
pages = {963--973},
url = {https://www.sciencedirect.com/science/article/pii/S1568494617305707}
}
|
| Torres ML, Olaso JM, Montenegro C, Santana R, Vazquez A, Justo R, Lozano JA, Schloegl S, Chollet G, Dugan N, Irvine M, Glackin N, Pickard C, Esposito A, Cordasco G, Troncone A, Petrovska-Delacretaz D, Mtibaa A, Hmani MA, Korsnes MS, Martinussen LJ, Escalera S, Palmero-Cantarino C, Deroo O, Gordeeva O, Tenerio-Laranga J, Gonzalez-Fraile E, Fernandez-Ruanova B and Gonzalez-Pinto A (2019), "The EMPATHIC Project: Mid-term Achievements", In Proceedings of the 12th Conference on PErvasive Technologies Related to Assistive Environments Conference (PETRA-19). , pp. 629-638. |
| Abstract: The goal of active aging is to promote changes in the elderly community so as to maintain an active, independent and socially-engaged lifestyle. Technological advancements currently provide the necessary tools to foster and monitor such processes. This paper reports on mid-term achievements of the European H2020 EMPATHIC project, which aims to research, innovate, explore and validate new interaction paradigms and platforms for future generations of personalized virtual coaches to assist the elderly and their carers to reach the active aging goal, in the vicinity of their home. The project focuses on evidence-based, user-validated research and integration of intelligent technology, and context sensing methods through automatic voice, eye and facial analysis, integrated with visual and spoken dialogue system capabilities. In this paper, we describe the current status of the system, with a special emphasis on its components and their integration, the creation of a Wizard of Oz platform, and findings gained from user interaction studies conducted throughout the first 18 months of the project. |
BibTeX:
@inproceedings{Torres_et_al:2019,
author = {M. L. Torres and J. M. Olaso and C. Montenegro and R. Santana and A. Vazquez and R. Justo and J. A. Lozano and S. Schloegl and G. Chollet and N. Dugan and M. Irvine and N. Glackin and C. Pickard and A. Esposito and G. Cordasco and A. Troncone and D. Petrovska-Delacretaz and A. Mtibaa and M. A. Hmani and M. S. Korsnes and L. J. Martinussen and S. Escalera and C. Palmero-Cantarino and O. Deroo and O. Gordeeva and J. Tenerio-Laranga and E. Gonzalez-Fraile and B. Fernandez-Ruanova and A. Gonzalez-Pinto},
title = {The EMPATHIC Project: Mid-term Achievements},
booktitle = {Proceedings of the 12th Conference on PErvasive Technologies Related to Assistive Environments Conference (PETRA-19)},
year = {2019},
pages = {629-638},
url = {https://dl.acm.org/doi/abs/10.1145/3316782.3322764}
}
|
| Vadillo J and Santana R (2019), "Universal Adversarial Examples in Speech Command Classification", CoRR. Vol. abs/1911.10182 |
| Abstract: Adversarial examples are inputs intentionally perturbed with the aim of forcing a machine learning model to produce a wrong prediction, while the changes are not easily detectable by a human. Although this topic has been intensively studied in the image domain, classification tasks in the audio domain have received less attention. In this paper we address the existence of universal perturbations for speech command classification. We provide evidence that universal attacks can be generated for speech command classification tasks, which are able to generalize across different models to a significant extent. Additionally, a novel analytical framework is proposed for the evaluation of universal perturbations under different levels of universality, demonstrating that the feasibility of generating effective perturbations decreases as the universality level increases. Finally, we propose a more detailed and rigorous framework to measure the amount of distortion introduced by the perturbations, demonstrating that the methods employed by convention are not realistic in audio-based problems. |
BibTeX:
@article{Vadillo_and_Santana:2019,
author = {Jon Vadillo and Roberto Santana},
title = {Universal Adversarial Examples in Speech Command Classification},
journal = {CoRR},
year = {2019},
volume = {abs/1911.10182},
url = {http://arxiv.org/abs/1911.10182}
}
|
| Vadillo J and Santana R (2020), "On the human evaluation of audio adversarial examples", CoRR. Vol. abs/2001.08444 |
| Abstract: Human-machine interaction is increasingly dependent on speech communication. Machine Learning models are usually applied to interpret human speech commands. However, these models can be fooled by adversarial examples, which are inputs intentionally perturbed to produce a wrong prediction without being noticed. While much research has been focused on developing new techniques to generate adversarial perturbations, less attention has been given to aspects that determine whether and how the perturbations are noticed by humans. This question is relevant since high fooling rates of proposed adversarial perturbation strategies are only valuable if the perturbations are not detectable. In this paper we investigate to which extent the distortion metrics proposed in the literature for audio adversarial examples, and which are commonly applied to evaluate the effectiveness of methods for generating these attacks, are a reliable measure of the human perception of the perturbations. Using an analytical framework, and an experiment in which 18 subjects evaluate audio adversarial examples, we demonstrate that the metrics employed by convention are not a reliable measure of the perceptual similarity of adversarial examples in the audio domai |
BibTeX:
@article{Vadillo_and_Santana:2020,
author = {Jon Vadillo and Roberto Santana},
title = {On the human evaluation of audio adversarial examples},
journal = {CoRR},
year = {2020},
volume = {abs/2001.08444},
url = {http://arxiv.org/abs/2001.08444}
}
|
| Vadillo J and Santana R (2022), "On the human evaluation of universal audio adversarial perturbations", Computers & Security. Vol. 112, pp. 102495. |
| Abstract: Human-machine interaction is increasingly dependent on speech communication, mainly due to the remarkable performance of Machine Learning models in speech recognition tasks. However, these models can be fooled by adversarial examples, which are inputs intentionally perturbed to produce a wrong prediction without the changes being noticeable to humans. While much research has focused on developing new techniques to generate adversarial perturbations, less attention has been given to aspects that determine whether and how the perturbations are noticed by humans. This question is relevant since high fooling rates of proposed adversarial perturbation strategies are only valuable if the perturbations are not detectable. In this paper we investigate to which extent the distortion metrics proposed in the literature for audio adversarial examples, and which are commonly applied to evaluate the effectiveness of methods for generating these attacks, are a reliable measure of the human perception of the perturbations. Using an analytical framework, and an experiment in which 36 subjects evaluate audio adversarial examples according to different factors, we demonstrate that the metrics employed by convention are not a reliable measure of the perceptual similarity of adversarial examples in the audio domain. |
BibTeX:
@article{Vadillo_and_Santana:2022,
author = {Jon Vadillo and Roberto Santana},
title = {On the human evaluation of universal audio adversarial perturbations},
journal = {Computers & Security},
year = {2022},
volume = {112},
pages = {102495},
url = {https://www.sciencedirect.com/science/article/pii/S0167404821003199},
doi = {10.1016/j.cose.2021.102495}
}
|
| Vadillo J, Santana R and Lozano JA (2020), "Analysis of Dominant Classes in Universal Adversarial Perturbations", CoRR. Vol. abs/2012.14352 |
| Abstract: The reasons why Deep Neural Networks are susceptible to being fooled by adversarial examples remains an open discussion. Indeed, many different strategies can be employed to efficiently generate adversarial attacks, some of them relying on different theoretical justifications. Among these strategies, universal (input-agnostic) perturbations are of particular interest, due to their capability to fool a network independently of the input in which the perturbation is applied. In this work, we investigate an intriguing phenomenon of universal perturbations, which has been reported previously in the literature, yet without a proven justification: universal perturbations change the predicted classes for most inputs into one particular (dominant) class, even if this behavior is not specified during the creation of the perturbation. In order to justify the cause of this phenomenon, we propose a number of hypotheses and experimentally test them using a speech command classification problem in the audio domain as a testbed. Our analyses reveal interesting properties of universal perturbations, suggest new methods to generate such attacks and provide an explanation of dominant classes, under both a geometric and a data-feature perspective. |
BibTeX:
@article{Vadillo_et_al:2020,
author = {Jon Vadillo and Roberto Santana and Jose A. Lozano},
title = {Analysis of Dominant Classes in Universal Adversarial Perturbations},
journal = {CoRR},
year = {2020},
volume = {abs/2012.14352},
url = {http://arxiv.org/abs/2012.14352}
}
|
| Vadillo J, Santana R and Lozano JA (2020), "Extending Adversarial Attacks to Produce Adversarial Class Probability Distributions", CoRR. Vol. abs/2004.06383 |
| Abstract: Despite the remarkable performance and generalization levels of deep learning models in a wide range of artificial intelligence tasks, it has been demonstrated that these models can be easily fooled by the addition of imperceptible yet malicious perturbations to natural inputs. These altered inputs are known in the literature as adversarial examples. In this paper, we propose a novel probabilistic framework to generalize and extend adversarial attacks in order to produce a desired probability distribution for the classes when we apply the attack method to a large number of inputs. This novel attack strategy provides the attacker with greater control over the target model, and increases the complexity of detecting that the model is being systematically attacked. We introduce four different strategies to efficiently generate such attacks, and illustrate our approach by extending multiple adversarial attack algorithms. We also experimentally validate our approach for the spoken command classification task, an exemplary machine learning problem in the audio domain. Our results demonstrate that we can closely approximate any probability distribution for the classes while maintaining a high fooling rate and by injecting imperceptible perturbations to the inputs. |
BibTeX:
@article{Vadillo_et_al:2020a,
author = {Jon Vadillo and Roberto Santana and Jose A. Lozano},
title = {Extending Adversarial Attacks to Produce Adversarial Class Probability Distributions},
journal = {CoRR},
year = {2020},
volume = {abs/2004.06383},
url = {http://arxiv.org/abs/2004.06383}
}
|
| Vadillo J, Santana R and Lozano JA (2020), "Exploring Gaps in DeepFool in Search of More Effective Adversarial Perturbations", In Proceedings of the Sixth International Conference on Machine Learning, Optimization, and Data Science (LOD-2020). Tuscany, Italy Vol. 12566, pp. 215-227. Springer, Cham. |
| Abstract: Adversarial examples are inputs subtly perturbed to produce a wrong prediction in machine learning models, while remaining perceptually similar to the original input. To find adversarial examples, some attack strategies rely on linear approximations of different properties of the models. This opens a number of questions related to the accuracy of such approximations. In this paper we focus on DeepFool, a state-of-the-art attack algorithm, which is based on efficiently approximating the decision space of the target classifier to find the minimal perturbation needed to fool the model. The objective of this paper is to analyze the feasibility of finding inaccuracies in the linear approximation of DeepFool, with the aim of studying whether they can be used to increase the effectiveness of the attack. We introduce two strategies to efficiently explore gaps in the approximation of the decision boundaries, and evaluate our approach in a speech command classification task. |
BibTeX:
@inproceedings{Vadillo_et_al:2020b,
author = {Jon Vadillo and Roberto Santana and Jose A. Lozano},
title = {Exploring Gaps in DeepFool in Search of More Effective Adversarial Perturbations},
booktitle = {Proceedings of the Sixth International Conference on Machine Learning, Optimization, and Data Science (LOD-2020)},
publisher = {Springer, Cham},
year = {2020},
volume = {12566},
pages = {215--227},
url = {https://link.springer.com/chapter/10.1007/978-3-030-64580-9_18}
}
|
| Vadillo J, Santana R and Lozano JA (2021), "When and How to Fool Explainable Models (and Humans) with Adversarial Examples", CoRR. Vol. abs/2107.01943 |
BibTeX:
@article{Vadillo_et_al:2021,
author = {Vadillo, Jon and Santana, Roberto and Lozano, Jose A},
title = {When and How to Fool Explainable Models (and Humans) with Adversarial Examples},
journal = {CoRR},
year = {2021},
volume = {abs/2107.01943},
url = {http://arxiv.org/abs/2107.01943}
}
|
| Vadillo J, Santana R and Lozano JA (2022), "Analysis of dominant classes in universal adversarial perturbations", Knowledge-Based Systems. Vol. 236, pp. 107719. |
| Abstract: The reasons why Deep Neural Networks are susceptible to being fooled by adversarial examples remains an open discussion. Indeed, many different strategies can be employed to efficiently generate adversarial attacks, some of them relying on different theoretical justifications. Among these strategies, universal (input-agnostic) perturbations are of particular interest, due to their capability to fool a network independently of the input in which the perturbation is applied. In this work, we investigate an intriguing phenomenon of universal perturbations, which has been reported previously in the literature, yet without a proven justification: universal perturbations change the predicted classes for most inputs into one particular (dominant) class, even if this behavior is not specified during the creation of the perturbation. In order to justify the cause of this phenomenon, we propose a number of hypotheses and experimentally test them using a speech command classification problem in the audio domain as a testbed. Our analyses reveal interesting properties of universal perturbations, suggest new methods to generate such attacks and provide an explanation of dominant classes, under both a geometric and a data-feature perspective. |
BibTeX:
@article{Vadillo_et_al:2022,
author = {Jon Vadillo and Roberto Santana and Jose A. Lozano},
title = {Analysis of dominant classes in universal adversarial perturbations},
journal = {Knowledge-Based Systems},
year = {2022},
volume = {236},
pages = {107719},
url = {https://www.sciencedirect.com/science/article/pii/S0950705121009643},
doi = {10.1016/j.knosys.2021.107719}
}
|
| Zangari-de-Souza M, Santana R, Pozo ATR and Mendiburu A (2015), "MOEA/D-GM: Using probabilistic graphical models in MOEA/D for solving combinatorial optimization problems", CoRR. Vol. abs/1511.05625 |
| Abstract: Evolutionary algorithms based on modeling the statistical dependencies (interactions) between the variables have been proposed to solve a wide range of complex problems. These algorithms learn and sample probabilistic graphical models able to encode and exploit the regularities of the problem. This paper investigates the effect of using probabilistic modeling techniques as a way to enhance the behavior of MOEA/D framework. MOEA/D is a decomposition based evolutionary algorithm that decomposes a multi-objective optimization problem (MOP) in a number of scalar single-objective subproblems and optimizes them in a collaborative manner. MOEA/D framework has been widely used to solve several MOPs. The proposed algorithm, MOEA/D using probabilistic Graphical Models (MOEA/D-GM) is able to instantiate both univariate and multi-variate probabilistic models for each subproblem. To validate the introduced framework algorithm, an experimental study is conducted on a multi-objective version of the deceptive function Trap5. The results show that the variant of the framework (MOEA/D-Tree), where tree models are learned from the matrices of the mutual information between the variables, is able to capture the structure of the problem. MOEA/D-Tree is able to achieve significantly better results than both MOEA/D using genetic operators and MOEA/D using univariate probability models, in terms of the approximation to the true Pareto front. |
BibTeX:
@article{Zangari_et_al:2015,
author = {Zangari-de-Souza, Murilo and Roberto Santana and Aurora Trinidad Ramirez Pozo and Alexander Mendiburu},
title = {MOEA/D-GM: Using probabilistic graphical models in MOEA/D for solving combinatorial optimization problems},
journal = {CoRR},
year = {2015},
volume = {abs/1511.05625},
url = {http://arxiv.org/abs/1511.05625}
}
|
| Zangari M, Santana R, Mendiburu A and Pozo A (2015), "PBIL: un mismo nombre para distintos algoritmos. un caso de estudio sobre un problema de optimización multi-objetivo", Proceedings of the Spanish Asociation in Artificial Inteligence (CAEPIA). , pp. 283-295. |
| Abstract: La optimización basada en el uso de técnicas que tratan de obtener un modelado probabilístico del espacio de búsqueda es una de las líneas de investigación que más ha avanzado en los últimos años en el ámbito de los Algoritmos Evolutivos. El algoritmo de aprendizaje incremental basado en poblaciones (PBIL) en una de las primeras propuestas introducidas en este campo, y ha sido ampliamente utilizado para resolver diferentes problemas de optimización. En este artículo queremos alertar de que las diferentes aplicaciones de PBIL publicadas en las literatura corresponden en realidad a dos implementaciones diferentes, en función de cómo ha sido definida la fase de aprendizaje del modelo. Estudiamos, analítica y empíricamente, el impacto que el método de aprendizaje utilizado tiene sobre el comportamiento del algoritmo. Como resultado de nuestra investigación, mostramos un caso de uso sobre un problema multi-objetivo, donde la elección de la variante PBIL puede producir resultados cualitativamente diferentes a lo largo del proceso de búsqueda. Utilizando diferentes versiones de PBIL como modelo base del algoritmo MOEA/D, evaluamos su efecto en la eficiencia de la búsqueda. |
BibTeX:
@inproceedings{Zangari_et_al:2015a,
author = {Zangari, M and Santana, R and Mendiburu, A and Pozo, A},
title = {PBIL: un mismo nombre para distintos algoritmos. un caso de estudio sobre un problema de optimización multi-objetivo},
journal = {Proceedings of the Spanish Asociation in Artificial Inteligence (CAEPIA)},
year = {2015},
pages = {283--295}
}
|
| Zangari-de-Souza M, Santana R, Mendiburu A and Pozo A (2016), "On the Design of Hard mUBQP Instances", In Proceedings of the 2016 on Genetic and Evolutionary Computation Conference. , pp. 421-428. |
| Abstract: This paper proposes a new method for the design and analysis of multi-objective unconstrained binary quadratic programming (mUBQP) instances, commonly used for testing discrete multi-objective evolutionary algorithms (MOEAs). These instances are usually generated considering the sparsity of the matrices and the correlation between objectives but randomly selecting the values for the matrix cells. Our hypothesis is that going beyond the use of objective correlations by considering different types of variables interactions in the generation of the instances can help to obtain more diverse problem benchmarks, comprising harder instances. We propose a parametric approach in which small building blocks of deceptive functions are planted into the matrices that define the mUBQP. The algorithm for creating the new instances is presented, and the difficulty of the functions is tested using variants of a decomposition-based MOEA. Our experimental results confirm that the instances generated by planting deceptive blocks require more function evaluations to be solved than instances generated using other methods. |
BibTeX:
@inproceedings{Zangari_et_al:2016,
author = {Zangari-de-Souza, Murilo and Santana, Roberto and Mendiburu, Alexander and Pozo, Aurora},
title = {On the Design of Hard mUBQP Instances},
booktitle = {Proceedings of the 2016 on Genetic and Evolutionary Computation Conference},
year = {2016},
pages = {421--428},
url = {https://dl.acm.org/doi/10.1145/2908812.2908889}
}
|
| de-Souza MZ, Santana R, Mendiburu A and Pozo ATR (2017), "Not all PBILs are the same: Unveiling the different learning mechanisms of PBIL variants", Applied Soft Computing. Vol. 53, pp. 88-96. |
| Abstract: Model-based optimization using probabilistic modeling of the search space is one of the areas where research on evolutionary algorithms (EAs) has considerably advanced in recent years. The population-based incremental algorithm (PBIL) is one of the first algorithms of its kind and it has been extensively applied to many optimization problems. In this paper we show that the different applications of PBIL reported in the literature correspond, in fact, to two essentially different algorithms, which are defined by the way the learning step is implemented. We analytically and empirically study the impact of the learning method on the search behavior of the algorithm. As a result of our research, we show examples in which the choice of a PBIL variant can produce qualitatively different outputs of the search process. |
BibTeX:
@article{Zangari_et_al:2017,
author = {Murilo Zangari-de-Souza and Roberto Santana and Alexander Mendiburu and Aurora Trinidad Ramirez Pozo},
title = {Not all PBILs are the same: Unveiling the different learning mechanisms of PBIL variants},
journal = {Applied Soft Computing},
year = {2017},
volume = {53},
pages = {88--96},
url = {https://www.sciencedirect.com/science/article/pii/S1568494616306743}
}
|
| Zangari-de-Souza M, Mendiburu A, Santana R and Pozo A (2017), "Multiobjective Decomposition-based Mallows Models Estimation of Distribution Algorithm. A case of study for Permutation Flowshop Scheduling Problem", Information Sciences. Vol. 397--398, pp. 137-154. Elsevier. |
| Abstract: Estimation of distribution algorithms (EDAs) have become a reliable alternative to solve a broad range of single and multi-objective optimization problems. Recently, distance-based exponential models, such as Mallows Model (MM) and Generalized Mallows Model (GMM), have demonstrated their validity in the context of EDAs to deal with permutation-based optimization problems. The aim of this paper is two-fold. First, we introduce a novel general multi-objective decomposition-based EDA using Kernels of Mallows models (MEDA/D-MK framework) for solving multi-objective permutation-based optimization problems. Second, in order to demonstrate the validity of the MEDA/D-MK, we have applied it to solve the multi-objective permutation flowshop scheduling problem (MoPFSP) minimizing the total flow time and the makespan. The permutation flowshop scheduling problem is one of the most studied problems of this kind due to its fields of application and algorithmic challenge. The results of our experiments show that MEDA/D-MK outperforms an improved MOEA/D variant specific tailored for minimizing makespan and total flowtime. Furthermore, our approach achieves competitive results compared to the best-known approximated Pareto fronts reported in the literature for the benchmark considered. |
BibTeX:
@article{Zangari_et_al:2017a,
author = {Zangari-de-Souza, Murilo and Mendiburu, Alexander and Santana, Roberto and Pozo, Aurora},
title = {Multiobjective Decomposition-based Mallows Models Estimation of Distribution Algorithm. A case of study for Permutation Flowshop Scheduling Problem},
journal = {Information Sciences},
publisher = {Elsevier},
year = {2017},
volume = {397--398},
pages = {137--154},
url = {https://www.sciencedirect.com/science/article/pii/S0020025517305352}
}
|
| Zangari-de-Souza M, Pozo A, Santana R and Mendiburu A (2017), "A decomposition-based binary ACO algorithm for the multiobjective UBQP", Neurocomputing. Vol. 246, pp. 58-68. Elsevier. |
| Abstract: The multiobjective unconstrained binary quadratic programming (mUBQP) is a combinatorial optimization problem which is able to represent several multiobjective optimization problems (MOPs). The problem can be characterized by the number of variables, the number of objectives and the objective correlation strength. Multiobjective evolutionary algorithms (MOEAs) are known as an efficient technique for solving MOPs. Moreover, several recent studies have shown the effectiveness of the MOEA/D framework applied to different MOPs. Previously, we have presented a preliminary study on an algorithm based on MOEA/D framework and the bio-inspired metaheuristic called binary ant colony optimization (BACO). The metaheuristic uses a positive feedback mechanism according to the best solutions found so far to update a probabilistic model which maintains the learned information. This paper presents the improved MOEA/D-BACO framework for solving the mUBQP. The components (i) mutation-like effect, and (ii) diversity preserving method are incorporated into the framework to enhance its search ability avoiding the premature convergence of the model and consequently maintaining a more diverse population of solutions. Experimental studies were conducted on a set of mUBQP instances. The results have shown that the proposed MOEA/D-BACO has outperformed MOEA/D, which uses genetic operators, in most of the test instances. Moreover, the algorithm has produced competitive results in comparison to the best approximated Pareto fronts from the literature. |
BibTeX:
@article{Zangari_et_al:2017b,
author = {Zangari-de-Souza, Murilo and Pozo, Aurora and Santana, Roberto and Mendiburu, Alexander},
title = {A decomposition-based binary ACO algorithm for the multiobjective UBQP},
journal = {Neurocomputing},
publisher = {Elsevier},
year = {2017},
volume = {246},
pages = {58--68},
url = {https://www.sciencedirect.com/science/article/pii/S0925231217302254}
}
|
| Zheng W-L, Santana R and Lu B-L (2015), "Comparison of Classification Methods for EEG-based Emotion Recognition", In Proceedings of the 2015 World Congress on Medical Physics and Biomedical Engineering. , pp. 1184-1187. Springer. |
| Abstract: In this paper, we review different classification methods for emotion recognition from EEG and perform a detailed comparison of these methods on a relatively larger dataset of 45 experiments. We propose to combine the classifiers using stacking to improve the emotion recognition accuracies. Experimental results show that the combination of classifiers using stacking can achieve higher average accuracies than that without stacking methods. The weights derived from the classifiers are investigated to extract the relevant features and present their biological interpretation as critical brain areas and critical frequency bands. |
BibTeX:
@inproceedings{Zheng_et_al:2015,
author = {W.-L Zheng and R. Santana and B.-L Lu},
title = {Comparison of Classification Methods for EEG-based Emotion Recognition},
booktitle = {Proceedings of the 2015 World Congress on Medical Physics and Biomedical Engineering},
publisher = {Springer},
year = {2015},
pages = {1184--1187},
url = {https://link.springer.com/chapter/10.1007/978-3-319-19387-8_287}
}
|