Distributed mixture-of-experts for Big Data using PETUUM framework

Billy Peralta, Luis Parra, Oriel Herrera, Luis Caro

Resultado de la investigación: Conference contribution

Resumen

Today, organizations are beginning to realize the importance of using as much data as possible for decision-making in their strategy. The finding of relevant patterns in enormous amount of data requires automatic machine learning algorithms, among them, a popular option is the mixture-of-experts that allows to model data using a set of local experts. The problem of using typical learning algorithms over Big Data is the handling of these large datasets in primary memory. In this paper, we propose a methodology to learn a mixture-of-experts in a distributed way using PETUUM platform. Particularly, we propose to learn the parameters of mixture-of-experts by adapting the standard stochastic gradient descent in a distributed way. This methodology is applied to people detection with standard real datasets considering accuracy and precision metrics among other. The results show a consistent performance of mixture-of-experts models where the best number of experts varies according to the particular dataset. We also evidence the advantages of the distributed approach by showing the almost linear decreasing of average training time according to the number of processors. In a future work, we expect to apply this methodology to mixture-of-experts with embedded variable selection.

Idioma originalEnglish
Título de la publicación alojada2017 36th International Conference of the Chilean Computer Science Society, SCCC 2017
EditorialIEEE Computer Society
Páginas1-7
Número de páginas7
Volumen2017-October
ISBN (versión digital)9781538634837
DOI
EstadoPublished - 5 jul 2018
Evento36th International Conference of the Chilean Computer Science Society, SCCC 2017 - Arica, Chile
Duración: 16 oct 201720 oct 2017

Conference

Conference36th International Conference of the Chilean Computer Science Society, SCCC 2017
PaísChile
CiudadArica
Período16/10/1720/10/17

Huella dactilar

Learning algorithms
Learning systems
Decision making
Big data
Data storage equipment

ASJC Scopus subject areas

  • Engineering(all)
  • Computer Science(all)

Citar esto

Peralta, B., Parra, L., Herrera, O., & Caro, L. (2018). Distributed mixture-of-experts for Big Data using PETUUM framework. En 2017 36th International Conference of the Chilean Computer Science Society, SCCC 2017 (Vol. 2017-October, pp. 1-7). IEEE Computer Society. https://doi.org/10.1109/SCCC.2017.8405113
Peralta, Billy ; Parra, Luis ; Herrera, Oriel ; Caro, Luis. / Distributed mixture-of-experts for Big Data using PETUUM framework. 2017 36th International Conference of the Chilean Computer Science Society, SCCC 2017. Vol. 2017-October IEEE Computer Society, 2018. pp. 1-7
@inproceedings{c28eed5d3f1c4b4fba2ce94ddf4a0997,
title = "Distributed mixture-of-experts for Big Data using PETUUM framework",
abstract = "Today, organizations are beginning to realize the importance of using as much data as possible for decision-making in their strategy. The finding of relevant patterns in enormous amount of data requires automatic machine learning algorithms, among them, a popular option is the mixture-of-experts that allows to model data using a set of local experts. The problem of using typical learning algorithms over Big Data is the handling of these large datasets in primary memory. In this paper, we propose a methodology to learn a mixture-of-experts in a distributed way using PETUUM platform. Particularly, we propose to learn the parameters of mixture-of-experts by adapting the standard stochastic gradient descent in a distributed way. This methodology is applied to people detection with standard real datasets considering accuracy and precision metrics among other. The results show a consistent performance of mixture-of-experts models where the best number of experts varies according to the particular dataset. We also evidence the advantages of the distributed approach by showing the almost linear decreasing of average training time according to the number of processors. In a future work, we expect to apply this methodology to mixture-of-experts with embedded variable selection.",
author = "Billy Peralta and Luis Parra and Oriel Herrera and Luis Caro",
year = "2018",
month = "7",
day = "5",
doi = "10.1109/SCCC.2017.8405113",
language = "English",
volume = "2017-October",
pages = "1--7",
booktitle = "2017 36th International Conference of the Chilean Computer Science Society, SCCC 2017",
publisher = "IEEE Computer Society",
address = "United States",

}

Peralta, B, Parra, L, Herrera, O & Caro, L 2018, Distributed mixture-of-experts for Big Data using PETUUM framework. En 2017 36th International Conference of the Chilean Computer Science Society, SCCC 2017. vol. 2017-October, IEEE Computer Society, pp. 1-7, 36th International Conference of the Chilean Computer Science Society, SCCC 2017, Arica, Chile, 16/10/17. https://doi.org/10.1109/SCCC.2017.8405113

Distributed mixture-of-experts for Big Data using PETUUM framework. / Peralta, Billy; Parra, Luis; Herrera, Oriel; Caro, Luis.

2017 36th International Conference of the Chilean Computer Science Society, SCCC 2017. Vol. 2017-October IEEE Computer Society, 2018. p. 1-7.

Resultado de la investigación: Conference contribution

TY - GEN

T1 - Distributed mixture-of-experts for Big Data using PETUUM framework

AU - Peralta, Billy

AU - Parra, Luis

AU - Herrera, Oriel

AU - Caro, Luis

PY - 2018/7/5

Y1 - 2018/7/5

N2 - Today, organizations are beginning to realize the importance of using as much data as possible for decision-making in their strategy. The finding of relevant patterns in enormous amount of data requires automatic machine learning algorithms, among them, a popular option is the mixture-of-experts that allows to model data using a set of local experts. The problem of using typical learning algorithms over Big Data is the handling of these large datasets in primary memory. In this paper, we propose a methodology to learn a mixture-of-experts in a distributed way using PETUUM platform. Particularly, we propose to learn the parameters of mixture-of-experts by adapting the standard stochastic gradient descent in a distributed way. This methodology is applied to people detection with standard real datasets considering accuracy and precision metrics among other. The results show a consistent performance of mixture-of-experts models where the best number of experts varies according to the particular dataset. We also evidence the advantages of the distributed approach by showing the almost linear decreasing of average training time according to the number of processors. In a future work, we expect to apply this methodology to mixture-of-experts with embedded variable selection.

AB - Today, organizations are beginning to realize the importance of using as much data as possible for decision-making in their strategy. The finding of relevant patterns in enormous amount of data requires automatic machine learning algorithms, among them, a popular option is the mixture-of-experts that allows to model data using a set of local experts. The problem of using typical learning algorithms over Big Data is the handling of these large datasets in primary memory. In this paper, we propose a methodology to learn a mixture-of-experts in a distributed way using PETUUM platform. Particularly, we propose to learn the parameters of mixture-of-experts by adapting the standard stochastic gradient descent in a distributed way. This methodology is applied to people detection with standard real datasets considering accuracy and precision metrics among other. The results show a consistent performance of mixture-of-experts models where the best number of experts varies according to the particular dataset. We also evidence the advantages of the distributed approach by showing the almost linear decreasing of average training time according to the number of processors. In a future work, we expect to apply this methodology to mixture-of-experts with embedded variable selection.

UR - http://www.scopus.com/inward/record.url?scp=85050966719&partnerID=8YFLogxK

U2 - 10.1109/SCCC.2017.8405113

DO - 10.1109/SCCC.2017.8405113

M3 - Conference contribution

AN - SCOPUS:85050966719

VL - 2017-October

SP - 1

EP - 7

BT - 2017 36th International Conference of the Chilean Computer Science Society, SCCC 2017

PB - IEEE Computer Society

ER -

Peralta B, Parra L, Herrera O, Caro L. Distributed mixture-of-experts for Big Data using PETUUM framework. En 2017 36th International Conference of the Chilean Computer Science Society, SCCC 2017. Vol. 2017-October. IEEE Computer Society. 2018. p. 1-7 https://doi.org/10.1109/SCCC.2017.8405113