A proposal for mixture of experts with entropic regularization

Billy Peralta, Ariel Saavedra, Luis Caro

Resultado de la investigación: Contribución a los tipos de informe/libroContribución a la conferencia

1 Cita (Scopus)

Resumen

In these days, there are a growing interest in pattern recognition for tasks as prediction of weather events, recommendation of the best route, intrusion detection or face detection. Each of these tasks can be modelled as classification problem, where a common alternative is to use an ensemble model of classification. A well-known example is given by Mixture-of-Experts model, which represents a probabilistic artificial neural network consisting of local experts classifiers weighted by a gate network, and whose combination creates an environment of competition among experts seeking to obtain patterns of the data source. We observe that this architecture assume that one gate influence only one data point, consequently the training can be misguided in real datasets where the data is better explained by multiple experts. In this work, we present a variant of regular Mixture-of-Experts model, which consists of maximizing of the entropy of gate network in addition to classification cost minimization. The results show the advantage of our approach in multiple datasets in terms of accuracy metric. As a future work, we plan to apply this idea to the Mixture-of-Experts with embedded feature selection.

Idioma originalInglés
Título de la publicación alojada2017 43rd Latin American Computer Conference, CLEI 2017
EditoresRodrigo Santos, Hector Monteverde
EditorialInstitute of Electrical and Electronics Engineers Inc.
Páginas1-9
Número de páginas9
Volumen2017-January
ISBN (versión digital)9781538630570
DOI
EstadoPublicada - 18 dic 2017
Evento43rd Latin American Computer Conference, CLEI 2017 - Cordoba, Argentina
Duración: 4 sep 20178 sep 2017

Conferencia

Conferencia43rd Latin American Computer Conference, CLEI 2017
PaísArgentina
CiudadCordoba
Período4/09/178/09/17

Áreas temáticas de ASJC Scopus

  • Redes de ordenadores y comunicaciones
  • Sistemas de información
  • Software
  • Educación

Huella Profundice en los temas de investigación de 'A proposal for mixture of experts with entropic regularization'. En conjunto forman una huella única.

  • Citar esto

    Peralta, B., Saavedra, A., & Caro, L. (2017). A proposal for mixture of experts with entropic regularization. En R. Santos, & H. Monteverde (Eds.), 2017 43rd Latin American Computer Conference, CLEI 2017 (Vol. 2017-January, pp. 1-9). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CLEI.2017.8226425