| Garcia I and Santana R (2021), "Unified Framework for the Analysis of the Effect of Control Policies on Automatic Voltage Regulators", TechRxiv. Vol. 15022611 |
| Abstract: With the advent of smart grids, voltage fluctuation has increased, especially in active distribution networks with a high penetration of distributed energy resources and a large deployment of electric vehicles. In this context, on-load tap-changer (OLTC) distribution transformers have become a key component, mainly because they provide automatic voltage regulation capability. In order to maximise the lifetime of OLTC devices, the number of tap operations should be minimised, avoiding unnecessary changes, but ensuring the main requirement: to keep the voltage within the limits permitted. Therefore, when the automatic mode is active, the control policy followed by the automatic voltage regulator is decisive. This paper presents a novel form of functional approximation of these policies. Furthermore, by means of a unified framework, a methodology for the simulation of policies based on control theory is proposed. The unified framework has been validated using real data. The results confirm the ability of the introduced framework to simulate different scenarios, optimising and validating both existing and new policies by observing their effect on transformer behavior. In addition, it allows the determination of the best-fit policies depending on characteristics such as the pre-selected voltage set point or the voltage variation between transformer taps. |
BibTeX:
@article{Garcia_and_Santana:2021,
author = {Garcia, Iker and Santana, Roberto},
title = {Unified Framework for the Analysis of the Effect of Control Policies on Automatic Voltage Regulators},
journal = {TechRxiv},
year = {2021},
volume = {15022611},
url = {https://www.techrxiv.org/articles/preprint/Unified_Framework_for_the_Analysis_of_the_Effect_of_Control_Policies_on_Automatic_Voltage_Regulators/15022611}
}
|
| Garciarena U, Mendiburu A and Santana R (2021), "Towards automatic construction of multi-network models for heterogeneous multi-task learning", ACM Transactions on Knowledge Discovery from Data (TKDD). Vol. 15(2), pp. 1-23. ACM New York, NY, USA. |
| Abstract: Multi-task learning, as it is understood nowadays, consists of using one single model to carry out several similar tasks. From classifying hand-written characters of different alphabets to figuring out how to play several Atari games using reinforcement learning, multi-task models have been able to widen their performance range across different tasks, although these tasks are usually of a similar nature. In this work, we attempt to expand this range even further, by including heterogeneous tasks in a single learning procedure. To do so, we firstly formally define a multi-network model, identifying the necessary components and characteristics to allow different adaptations of said model depending on the tasks it is required to fulfill. Secondly, employing the formal definition as a starting point, we develop an illustrative model example consisting of three different tasks (classification, regression, and data sampling). The performance of this illustrative model is then analyzed, showing its capabilities. Motivated by the results of the analysis, we enumerate a set of open challenges and future research lines over which the full potential of the proposed model definition can be exploited. |
BibTeX:
@article{Garciarena_et_al:2021,
author = {Garciarena, Unai and Mendiburu, Alexander and Santana, Roberto},
title = {Towards automatic construction of multi-network models for heterogeneous multi-task learning},
journal = {ACM Transactions on Knowledge Discovery from Data (TKDD)},
publisher = {ACM New York, NY, USA},
year = {2021},
volume = {15},
number = {2},
pages = {1--23},
url = {https://dl.acm.org/doi/abs/10.1145/3434748}
}
|
| Garciarena U, Santana R and Mendiburu A (2021), "Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components", CoRR. Vol. abs/2106.08972 |
| Abstract: With neural architecture search methods gaining ground on manually designed deep neural networks -even more rapidly as model sophistication escalates-, the research trend shifts towards arranging different and often increasingly complex neural architecture search spaces. In this conjuncture, delineating algorithms which can efficiently explore these search spaces can result in a significant improvement over currently used methods, which, in general, randomly select the structural variation operator, hoping for a performance gain. In this paper, we investigate the effect of different variation operators in a complex domain, that of multi-network heterogeneous neural models. These models have an extensive and complex search space of structures as they require multiple sub-networks within the general model in order to answer to different output types. From that investigation, we extract a set of general guidelines, whose application is not limited to that particular type of model, and are useful to determine the direction in which an architecture optimization method could find the largest improvement. To deduce the set of guidelines, we characterize both the variation operators, according to their effect on the complexity and performance of the model; and the models, relying on diverse metrics which estimate the quality of the different parts composing it. |
BibTeX:
@article{Garciarena_et_al:2021b,
author = {Garciarena, Unai and Santana, Roberto and Mendiburu, Alexander},
title = {Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components},
journal = {CoRR},
year = {2021},
volume = {abs/2106.08972},
url = {http://arxiv.org/abs/2106.08972}
}
|
| Garciarena U, Lourenço N, Machado P, Santana R and Mendiburu A (2021), "On the Exploitation of Neuroevolutionary Information: Analyzing the Past for a More Efficient Future", CoRR. Vol. abs/2105.12836 |
| Abstract: Neuroevolutionary algorithms, automatic searches of neural network structures by means of evolutionary techniques, are computationally costly procedures. In spite of this, due to the great performance provided by the architectures which are found, these methods are widely applied. The final outcome of neuroevolutionary processes is the best structure found during the search, and the rest of the procedure is commonly omitted in the literature. However, a good amount of residual information consisting of valuable knowledge that can be extracted is also produced during these searches. In this paper, we propose an approach that extracts this information from neuroevolutionary runs, and use it to build a metamodel that could positively impact future neural architecture searches. More specifically, by inspecting the best structures found during neuroevolutionary searches of generative adversarial networks with varying characteristics (e.g., based on dense or convolutional layers), we propose a Bayesian network-based model which can be used to either find strong neural structures right away, conveniently initialize different structural searches for different problems, or help future optimization of structures of any type to keep finding increasingly better structures where uninformed methods get stuck into local optima. |
BibTeX:
@article{Garciarena_et_al:2021c,
author = {Garciarena, Unai and Lourenço, Nuno and Machado, Penousal and Santana, Roberto and Mendiburu, Alexander},
title = {On the Exploitation of Neuroevolutionary Information: Analyzing the Past for a More Efficient Future},
journal = {CoRR},
year = {2021},
volume = {abs/2105.12836},
url = {http://arxiv.org/abs/2105.12836}
}
|
| Garciarena U, Vadillo J, Mendiburu A and Santana R (2021), "Adversarial Perturbations for Evolutionary Optimization", In International Conference on Machine Learning, Optimization, and Data Science (LOD-2021). Vol. 13164, pp. 408-422. |
| Abstract: Sampling methods are a critical step for model-based evolutionary algorithms, their goal being the generation of new and promising individuals based on the information provided by the model. Adversarial perturbations have been proposed as a way to create samples that deceive neural networks. In this paper we introduce the idea of creating adversarial perturbations that correspond to promising solutions of the search space. A surrogate neural network is “fooled” by an adversarial perturbation algorithm until it produces solutions that are likely to be of higher fitness than the present ones. Using a benchmark of functions with varying levels of difficulty, we investigate the performance of a number of adversarial perturbation techniques as sampling methods. The paper also proposes a technique to enhance the effect that adversarial perturbations produce in the network. While adversarial perturbations on their own are not able to produce evolutionary algorithms that compete with state of the art methods, they provide a novel and promising way to combine local optimizers with evolutionary algorithms. |
BibTeX:
@inproceedings{Garciarena_et_al:2021d,
author = {Garciarena, U. and Vadillo, J. and Mendiburu, A. and Santana, R.},
title = {Adversarial Perturbations for Evolutionary Optimization},
booktitle = {International Conference on Machine Learning, Optimization, and Data Science (LOD-2021)},
year = {2021},
volume = {13164},
pages = {408--422},
url = {https://link.springer.com/chapter/10.1007/978-3-030-95470-3_31}
}
|
| Garcia Rodriguez MJ, Rodriguez Montequin V, Aranguren Ubierna A, Santana R, Sierra Araujo B and Zelaia Jauregi A (2021), "Award Price Estimator for Public Procurement Auctions Using Machine Learning Algorithms: Case Study with Tenders from Spain", Studies in Informatics and Control. Vol. 30(4), pp. 67-76. |
| Abstract: The public procurement process plays an important role in the efficient use of public resources. In this context, the evaluation of machine learning techniques that are able to predict the award price is a relevant research topic. In this paper, the suitability of a representative set of machine learning algorithms is evaluated for this problem. The traditional regression methods, such as linear regression and random forest, are compared with the less investigated paradigms, such as isotonic regression and popular artificial neural network models. Extensive experiments are conducted based on the Spanish public procurement announcements (tenders) dataset and employ diverse error metrics and implementations in WEKA and Tensorflow 2. |
BibTeX:
@article{GarciaRodriguez_et_al:2021,
author = {Garcia Rodriguez, Manuel J and Rodriguez Montequin, Vicente and Aranguren Ubierna, Andoni and Santana, Roberto and Sierra Araujo, Basilio and Zelaia Jauregi, Ana},
title = {Award Price Estimator for Public Procurement Auctions Using Machine Learning Algorithms: Case Study with Tenders from Spain},
journal = {Studies in Informatics and Control},
year = {2021},
volume = {30},
number = {4},
pages = {67--76},
url = {https://sic.ici.ro/award-price-estimator-for-public-procurement-auctions-using-machine-learning-algorithms-case-study-with-tenders-from-spain/}
}
|
| Arenas ZG, Jimenez JC, Lozada-Chang L-V and Santana R (2021), "Estimation of distribution algorithms for the computation of innovation estimators of diffusion processes", Mathematics and Computers in Simulation. Vol. 187, pp. 449-467. Elsevier. |
| Abstract: Innovation Method is a recognized method for the estimation of parameters in diffusion processes. It is well known that the quality of the Innovation Estimator strongly depends on an adequate selection of the initial value for the parameters when a local optimization algorithm is used in its computation. Alternatively, in this paper, we use a strategy based on a modern method for solving global optimization problems, Estimation of Distribution Algorithms (EDAs). We study the feasibility of a specific EDA - a continuous version of the Univariate Marginal Distribution Algorithm (UMDAc) - for the computation of the Innovation Estimators. Through numerical simulations, we show that the considered global optimization algorithms substantially improves the effectiveness of the Innovation Estimators for different types of diffusion processes with complex nonlinear and stochastic dynamics. |
BibTeX:
@article{GonzalezArenas_et_al:2021,
author = {Arenas, Zochil González and Jimenez, Juan Carlos and Lozada-Chang, Li-Vang and Santana, Roberto},
title = {Estimation of distribution algorithms for the computation of innovation estimators of diffusion processes},
journal = {Mathematics and Computers in Simulation},
publisher = {Elsevier},
year = {2021},
volume = {187},
pages = {449--467},
url = {https://www.sciencedirect.com/science/article/pii/S0378475421000926}
}
|
| Lima RHR, Fontoura V, Pozo ATR, Mendiburu A and Santana R (2021), "Automatic Design of Deep Neural Networks Applied to Image Segmentation Problems", In Genetic Programming: 24th European Conference, EuroGP 2021, Held as Part of EvoStar 2021, Virtual Event, April 7--9, 2021, Proceedings. , pp. 98-112. |
| Abstract: A U-Net is a convolutional neural network mainly used for image segmentation domains such as medical image analysis. As other deep neural networks, the U-Net architecture influences the efficiency and accuracy of the network. We propose the use of a grammar-based evolutionary algorithm for the automatic design of deep neural networks for image segmentation tasks. The approach used is called Dynamic Structured Grammatical Evolution (DSGE), which employs a grammar to define the building blocks that are used to compose the networks, as well as the rules that help build them. We perform a set of experiments on the BSDS500 and ISBI12 datasets, designing networks tuned to image segmentation and edge detection. Subsequently, by using image similarity metrics, the results of our best performing networks are compared with the original U-Net. The results show that the proposed approach is able to design a network that is less complex in the number of trainable parameters, while also achieving slightly better results than the U-Net with a more consistent training. |
BibTeX:
@inproceedings{Lima_et_al:2021c,
author = {R. H. R. Lima and V. Fontoura and A. T. R. Pozo and A. Mendiburu and R. Santana},
title = {Automatic Design of Deep Neural Networks Applied to Image Segmentation Problems},
booktitle = {Genetic Programming: 24th European Conference, EuroGP 2021, Held as Part of EvoStar 2021, Virtual Event, April 7--9, 2021, Proceedings},
year = {2021},
pages = {98-112},
url = {https://www.springerprofessional.de/automatic-design-of-deep-neural-networks-applied-to-image-segmen/19000302}
}
|
| Martins MS, Yafrani ME, Delgado M, Lüders R, Santana R, Siqueira HV, Akcay HG and Ahiod B (2021), "Analysis of Bayesian Network Learning Techniques for a Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm: a case study on MNK Landscape", Journal of Heuristics. Vol. 27(4), pp. 549-573. Springer. |
| Abstract: This work investigates different Bayesian network structure learning techniques by thoroughly studying several variants of Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm (HMOBEDA), applied to the MNK Landscape combinatorial problem. In the experiments, we evaluate the performance considering three different aspects: optimization abilities, robustness and learning efficiency. Results for instances of multi- and many-objective MNK-landscape show that, score-based structure learning algorithms appear to be the best choice. In particular, HMOBEDA𝑘2 was capable of producing results comparable with the other variants in terms of the runtime of convergence and the coverage of the final Pareto front, with the additional advantage of providing solutions that are less sensible to noise while the variability of the corresponding Bayesian network models is reduced. |
BibTeX:
@article{Martins_et_al:2021,
author = {Martins, Marcella SR and Yafrani, Mohamed El and Delgado, Myriam and Lüders, Ricardo and Santana, Roberto and Siqueira, Hugo V and Akcay, Huseyin G and Ahiod, Bela\id},
title = {Analysis of Bayesian Network Learning Techniques for a Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm: a case study on MNK Landscape},
journal = {Journal of Heuristics},
publisher = {Springer},
year = {2021},
volume = {27},
number = {4},
pages = {549--573},
url = {https://link.springer.com/article/10.1007/s10732-021-09469-x}
}
|
| Mei N, Santana R and Soto D (2021), "Informative neural representations of unseen objects during higher-order processing in human brains and deep artificial networks", bioRxiv. Cold Spring Harbor Laboratory. |
| Abstract: Despite advances in the neuroscience of visual consciousness over the last decades, we still lack a framework for understanding the scope of unconscious processing and how it relates to conscious experience. Previous research observed brain signatures of unconscious contents in visual cortex, but these have not been identified in a reliable manner, with low trial numbers and signal detection theoretic constraints not allowing to decisively discard conscious perception. Critically, the extent to which unconscious content is represented in high-level processing stages along the ventral visual stream and linked prefrontal areas remains unknown. Using a within-subject, high-precision, highly-sampled fMRI approach, we show that unconscious contents, even those associated with null sensitivity, can be reliably decoded from multivoxel patterns that are highly distributed along the ventral visual pathway and also involving prefrontal substrates. Notably, the neural representation in these areas generalised across conscious and unconscious visual processing states, placing constraints on prior findings that fronto-parietal substrates support the representation of conscious contents and suggesting revisions to models of consciousness such as the neuronal global workspace. We then provide a computational model simulation of visual information processing/representation in the absence of perceptual sensitivity by using feedforward convolutional neural networks trained to perform a similar visual task to the human observers. The work provides a novel framework for pinpointing the neural representation of unconscious knowledge across different task domains. |
BibTeX:
@article{Mei_et_al:2021,
author = {Mei, Ning and Santana, Roberto and Soto, David},
title = {Informative neural representations of unseen objects during higher-order processing in human brains and deep artificial networks},
journal = {bioRxiv},
publisher = {Cold Spring Harbor Laboratory},
year = {2021},
url = {https://www.biorxiv.org/content/10.1101/2021.01.12.426428v1.abstract}
}
|
| Montenegro C, Santana R and Lozano JA (2021), "Analysis of the sensitivity of the End-Of-Turn Detection task to errors generated by the Automatic Speech Recognition process", Engineering Applications of Artificial Intelligence. Vol. 100, pp. 104189. Elsevier. |
| Abstract: An End-Of-Turn Detection Module (EOTD-M) is an essential component of automatic Spoken Dialogue Systems. The capability of correctly detecting whether a user’s utterance has ended or not improves the accuracy in interpreting the meaning of the message and decreases the latency in the answer. Usually, in dialogue systems, an EOTD-M is coupled with an Automatic Speech Recognition Module (ASR-M) to transmit complete utterances to the Natural Language Understanding unit. Mistakes in the ASR-M transcription can have a strong effect on the performance of the EOTD-M. The actual extent of this effect depends on the particular combination of ASR-M transcription errors and the sentence featurization techniques implemented as part of the EOTD-M. In this paper we investigate this important relationship for an EOTD-M based on semantic information and particular characteristics of the speakers (speech profiles). We introduce an Automatic Speech Recognition Simulator (ASR-SIM) that models different types of semantic mistakes in the ASR-M transcription as well as different speech profiles. We use the simulator to evaluate the sensitivity to ASR-M mistakes of a Long Short-Term Memory network classifier trained in EOTD with different featurization techniques. Our experiments reveal the different ways in which the performance of the model is influenced by the ASR-M errors. We corroborate that not only is the ASR-SIM useful to estimate the performance of an EOTD-M in customized noisy scenarios, but it can also be used to generate training datasets with the expected error rates of real working conditions, which leads to better performance. |
BibTeX:
@article{Montenegro_et_al:2021,
author = {C. Montenegro and R. Santana and J. A. Lozano},
title = {Analysis of the sensitivity of the End-Of-Turn Detection task to errors generated by the Automatic Speech Recognition process},
journal = {Engineering Applications of Artificial Intelligence},
publisher = {Elsevier},
year = {2021},
volume = {100},
pages = {104189},
url = {https://www.sciencedirect.com/science/article/pii/S0952197621000361}
}
|
| Olaso JM, Vázquez A, Ben Letaifa L, De Velasco M, Mtibaa A, Hmani MA, Petrovska-Delacrétaz D, Chollet G, Montenegro C, López-Zorrilla A, Justo R, Santana R and others (2021), "The EMPATHIC Virtual Coach: a demo", In Proceedings of the 2021 International Conference on Multimodal Interaction. , pp. 848-851. |
| Abstract: The main objective of the EMPATHIC project has been the design and development of a virtual coach to engage the healthy-senior user and to enhance well-being through awareness of personal status. The EMPATHIC approach addresses this objective through multimodal interactions supported by the GROW coaching model. The paper summarizes the main components of the EMPATHIC Virtual Coach (EMPATHIC-VC) and introduces a demonstration of the coaching sessions in selected scenarios. |
BibTeX:
@inproceedings{Olaso_et_al:2021,
author = {Olaso, Javier M and Vázquez, Alain and Ben Letaifa, Leila and De Velasco, Mikel and Mtibaa, Aymen and Hmani, Mohamed Amine and Petrovska-Delacrétaz, Dijana and Chollet, Gérard and Montenegro, César and López-Zorrilla, Asier and Justo, Raquel and Santana, Roberto and others},
title = {The EMPATHIC Virtual Coach: a demo},
booktitle = {Proceedings of the 2021 International Conference on Multimodal Interaction},
year = {2021},
pages = {848--851},
url = {https://dl.acm.org/doi/abs/10.1145/3462244.3481574}
}
|
| Roman I, Santana R, Mendiburu A and Lozano JA (2021), "Evolution of Gaussian Process kernels for machine translation post-editing effort estimation", Annals of Mathematics and Artificial Intelligence. Vol. 89, pp. 835-856. Springer. |
| Abstract: In many Natural Language Processing problems the combination of machine learning and optimization techniques is essential. One of these problems is the estimation of the human effort needed to improve a text that has been translated using a machine translation method. Recent advances in this area have shown that Gaussian Processes can be effective in post-editing effort prediction. However, Gaussian Processes require a kernel function to be defined, the choice of which highly influences the quality of the prediction. On the other hand, the extraction of features from the text can be very labor-intensive, although recent advances in sentence embedding have shown that this process can be automated. In this paper, we use a Genetic Programming algorithm to evolve kernels for Gaussian Processes to predict post-editing effort based on sentence embeddings. We show that the combination of evolutionary optimization and Gaussian Processes removes the need for a-priori specification of the kernel choice, and, by using a multi-objective variant of the Genetic Programming approach, kernels that are suitable for predicting several metrics can be learned. We also investigate the effect that the choice of the sentence embedding method has on the kernel learning process. |
BibTeX:
@article{Roman_et_al:2021,
author = {Roman, Ibai and Santana, Roberto and Mendiburu, Alexander and Lozano, Jose A},
title = {Evolution of Gaussian Process kernels for machine translation post-editing effort estimation},
journal = {Annals of Mathematics and Artificial Intelligence},
publisher = {Springer},
year = {2021},
volume = {89},
pages = {835-856},
url = {https://link.springer.com/article/10.1007/s10472-021-09751-5}
}
|
| Santana R (2021), "Semantic Composition of Word-Embeddings with Genetic Programming", In Heuristics for Optimization and Learning. , pp. 409-423. Springer, Cham. |
| Abstract: Word-embeddings are vectorized numerical representations of words increasingly applied in natural language processing. Spaces that comprise the embedding representations can capture semantic and other relationships between the words. In this paper we show that it is possible to learn methods for word composition in semantic spaces using genetic programming (GP). We propose to address the creation of word embeddings that have a target semantic content as an automatic program generation problem. We solve this problem using GP. Using a word analogy task as benchmark, we also show that GP-generated programs are able to obtain accuracy values above those produced by the commonly used human-designed rule for algebraic manipulation of word vectors. Finally, we show the robustness of our approach by executing the evolved programs on the word2vec GoogleNews vectors, learned over 3 billion running words, and assessing their accuracy in the same word analogy task. |
BibTeX:
@incollection{Santana:2021,
author = {Santana, R},
title = {Semantic Composition of Word-Embeddings with Genetic Programming},
booktitle = {Heuristics for Optimization and Learning},
publisher = {Springer, Cham.},
year = {2021},
pages = {409--423},
url = {https://link.springer.com/chapter/10.1007/978-3-030-58930-1_27}
}
|
| Vadillo J, Santana R and Lozano JA (2021), "When and How to Fool Explainable Models (and Humans) with Adversarial Examples", CoRR. Vol. abs/2107.01943 |
BibTeX:
@article{Vadillo_et_al:2021,
author = {Vadillo, Jon and Santana, Roberto and Lozano, Jose A},
title = {When and How to Fool Explainable Models (and Humans) with Adversarial Examples},
journal = {CoRR},
year = {2021},
volume = {abs/2107.01943},
url = {http://arxiv.org/abs/2107.01943}
}
|