Timed Dataflow: Reducing Communication Overhead for Distributed Machine Learning Systems

2016 ◽  
Author(s):  
Peng Sun ◽  
Yonggang Wen ◽  
Ta Nguyen Binh Duong ◽  
Shengen Yan
Keyword(s):  
Machine Learning ◽  
Learning Systems ◽  
Communication Overhead ◽  
Distributed Machine Learning

scholarly journals ArchNet: A data hiding design for distributed machine learning systems

2020 ◽  
pp. 101912
Author(s):  
Kaiyan Chang ◽  
Wei Jiang ◽  
Jinyu Zhan ◽  
Zicheng Gong ◽  
Weijia Pan
Keyword(s):  
Machine Learning ◽  
Data Hiding ◽  
Learning Systems ◽  
Distributed Machine Learning

scholarly journals Communication-efficient and Scalable Decentralized Federated Edge Learning

2021 ◽  
Author(s):  
Austine Zong Han Yapp ◽  
Hong Soo Nicholas Koh ◽  
Yan Ting Lai ◽  
Jiawen Kang ◽  
Xuandi Li ◽  
...  
Keyword(s):  
Machine Learning ◽  
Data Privacy ◽  
Large Scale ◽  
Local Model ◽  
Communication Overhead ◽  
Model Training ◽  
Collaborative Training ◽  
Distributed Machine Learning ◽  
The Coordinator

Federated Edge Learning (FEL) is a distributed Machine Learning (ML) framework for collaborative training on edge devices. FEL improves data privacy over traditional centralized ML model training by keeping data on the devices and only sending local model updates to a central coordinator for aggregation. However, challenges still remain in existing FEL architectures where there is high communication overhead between edge devices and the coordinator. In this paper, we present a working prototype of blockchain-empowered and communication-efficient FEL framework, which enhances the security and scalability towards large-scale implementation of FEL.

scholarly journals Blockchain for federated learning toward secure distributed machine learning systems: a systemic survey

2021 ◽  
Author(s):  
Dun Li ◽  
Dezhi Han ◽  
Tien-Hsiung Weng ◽  
Zibin Zheng ◽  
Hongzhi Li ◽  
...  
Keyword(s):  
Machine Learning ◽  
Learning Systems ◽  
Distributed Machine Learning

Tensor relational algebra for distributed machine learning system design

2021 ◽  
Vol 14 (8) ◽  
pp. 1338-1350
Author(s):  
Binhang Yuan ◽  
Dimitrije Jankov ◽  
Jia Zou ◽  
Yuxin Tang ◽  
Daniel Bourgeois ◽  
...  
Keyword(s):  
Machine Learning ◽  
Empirical Study ◽  
System Design ◽  
High Efficiency ◽  
Learning Systems ◽  
Learning System ◽  
Relational Algebra ◽  
Activation Functions ◽  
Distributed Environment ◽  
Distributed Machine Learning

We consider the question: what is the abstraction that should be implemented by the computational engine of a machine learning system? Current machine learning systems typically push whole tensors through a series of compute kernels such as matrix multiplications or activation functions, where each kernel runs on an AI accelerator (ASIC) such as a GPU. This implementation abstraction provides little built-in support for ML systems to scale past a single machine, or for handling large models with matrices or tensors that do not easily fit into the RAM of an ASIC. In this paper, we present an alternative implementation abstraction called the tensor relational algebra (TRA). The TRA is a set-based algebra based on the relational algebra. Expressions in the TRA operate over binary tensor relations, where keys are multi-dimensional arrays and values are tensors. The TRA is easily executed with high efficiency in a parallel or distributed environment, and amenable to automatic optimization. Our empirical study shows that the optimized TRA-based back-end can significantly outperform alternatives for running ML workflows in distributed clusters.

Platform for Analysing and Encouraging Student Activity on Contest and E-learning Systems

2018 ◽  
Vol 12 ◽  
pp. 85-98
Author(s):  
Bojan Kostadinov ◽  
Mile Jovanov ◽  
Emil STANKOV
Keyword(s):  
Machine Learning ◽  
Data Collection ◽  
Educational Policy ◽  
Learning Systems ◽  
Data Sources ◽  
Or Education ◽  
Student Activity ◽  
The World ◽  
E Learning ◽  
Analyse Data

Data collection and machine learning are changing the world. Whether it is medicine, sports or education, companies and institutions are investing a lot of time and money in systems that gather, process and analyse data. Likewise, to improve competitiveness, a lot of countries are making changes to their educational policy by supporting STEM disciplines. Therefore, it’s important to put effort into using various data sources to help students succeed in STEM. In this paper, we present a platform that can analyse student’s activity on various contest and e-learning systems, combine and process the data, and then present it in various ways that are easy to understand. This in turn enables teachers and organizers to recognize talented and hardworking students, identify issues, and/or motivate students to practice and work on areas where they’re weaker.

Sign in / Sign up

Export Citation Format

Share Document