Robust expectation maximization learning algorithm for mixture of experts

Romina Torres, Rodrigo Salas, Hector Allende, Claudio Moraga

Resultado de la investigación: Article

5 Citas (Scopus)

Resumen

The Mixture of Experts (ME) model is a type of modular artificial neural network (MANN) specially suitable when the search space is stratified and whose architecture is composed by different kinds of networks which compete to learn several aspects of a complex problem. Training a ME architecture can be treated as a maximum likelihood estimation problem, where the Expectation Maximization (EM) algorithm decouples the estimation process in a manner that fits well with the modular structure of the ME architecture. However, the learning process relies on the data and so is the performance. When the data is exposed to outliers, the model is affected by being sensible to these deviations obtaining a poor performance as it is shown in this work. This paper proposes a Robust Expectation Maximization algorithm for learning a ME model (REM-ME) based on M-estimators. We show empirically that the REM-ME for these architectures prevents performance deterioration due to outliers and yields significantly faster convergence than other approaches.

Idioma originalEnglish
Páginas (desde-hasta)238-245
Número de páginas8
PublicaciónLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volumen2686
EstadoPublished - 2003

Huella dactilar

Mixture of Experts
Expectation-maximization Algorithm
Learning algorithms
Learning Algorithm
Learning
Outlier
Modular Neural Networks
M-estimator
Maximum likelihood estimation
Robust Algorithm
Deterioration
Learning Process
Maximum Likelihood Estimation
Search Space
Artificial Neural Network
Deviation
Model
Neural networks
Architecture

ASJC Scopus subject areas

  • Computer Science(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Theoretical Computer Science

Citar esto

@article{31a4f98bb8c7446cac64d15bca2ee12c,
title = "Robust expectation maximization learning algorithm for mixture of experts",
abstract = "The Mixture of Experts (ME) model is a type of modular artificial neural network (MANN) specially suitable when the search space is stratified and whose architecture is composed by different kinds of networks which compete to learn several aspects of a complex problem. Training a ME architecture can be treated as a maximum likelihood estimation problem, where the Expectation Maximization (EM) algorithm decouples the estimation process in a manner that fits well with the modular structure of the ME architecture. However, the learning process relies on the data and so is the performance. When the data is exposed to outliers, the model is affected by being sensible to these deviations obtaining a poor performance as it is shown in this work. This paper proposes a Robust Expectation Maximization algorithm for learning a ME model (REM-ME) based on M-estimators. We show empirically that the REM-ME for these architectures prevents performance deterioration due to outliers and yields significantly faster convergence than other approaches.",
keywords = "Expectation Maximization, Mixtures of Experts, Modular Neural Networks, Robust Learning Algorithm",
author = "Romina Torres and Rodrigo Salas and Hector Allende and Claudio Moraga",
year = "2003",
language = "English",
volume = "2686",
pages = "238--245",
journal = "Lecture Notes in Computer Science",
issn = "0302-9743",
publisher = "Springer Verlag",

}

TY - JOUR

T1 - Robust expectation maximization learning algorithm for mixture of experts

AU - Torres, Romina

AU - Salas, Rodrigo

AU - Allende, Hector

AU - Moraga, Claudio

PY - 2003

Y1 - 2003

N2 - The Mixture of Experts (ME) model is a type of modular artificial neural network (MANN) specially suitable when the search space is stratified and whose architecture is composed by different kinds of networks which compete to learn several aspects of a complex problem. Training a ME architecture can be treated as a maximum likelihood estimation problem, where the Expectation Maximization (EM) algorithm decouples the estimation process in a manner that fits well with the modular structure of the ME architecture. However, the learning process relies on the data and so is the performance. When the data is exposed to outliers, the model is affected by being sensible to these deviations obtaining a poor performance as it is shown in this work. This paper proposes a Robust Expectation Maximization algorithm for learning a ME model (REM-ME) based on M-estimators. We show empirically that the REM-ME for these architectures prevents performance deterioration due to outliers and yields significantly faster convergence than other approaches.

AB - The Mixture of Experts (ME) model is a type of modular artificial neural network (MANN) specially suitable when the search space is stratified and whose architecture is composed by different kinds of networks which compete to learn several aspects of a complex problem. Training a ME architecture can be treated as a maximum likelihood estimation problem, where the Expectation Maximization (EM) algorithm decouples the estimation process in a manner that fits well with the modular structure of the ME architecture. However, the learning process relies on the data and so is the performance. When the data is exposed to outliers, the model is affected by being sensible to these deviations obtaining a poor performance as it is shown in this work. This paper proposes a Robust Expectation Maximization algorithm for learning a ME model (REM-ME) based on M-estimators. We show empirically that the REM-ME for these architectures prevents performance deterioration due to outliers and yields significantly faster convergence than other approaches.

KW - Expectation Maximization

KW - Mixtures of Experts

KW - Modular Neural Networks

KW - Robust Learning Algorithm

UR - http://www.scopus.com/inward/record.url?scp=21644462102&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:21644462102

VL - 2686

SP - 238

EP - 245

JO - Lecture Notes in Computer Science

JF - Lecture Notes in Computer Science

SN - 0302-9743

ER -