Mediaspace scheduled maintenance: Aug 25, 2026 07:00 - 12:00 AM. During this time, videos will be temporarily unavailable. Check status updates.

Concept

Apache Spark

Related publications (22)

ProGAP: Progressive Graph Neural Networks with Differential Privacy Guarantees

Daniel Gatica-Perez, Sina Sajadmanesh

Graph Neural Networks (GNNs) have become a popular tool for learning on graphs, but their widespread use raises privacy concerns as graph data can contain personal or sensitive information. Differentially private GNN models have been recently proposed to p ...

Assoc Computing Machinery2024

Task-driven neural network models predict neural dynamics of proprioception: Experimental data, activations and predictions of neural network models

Alexander Mathis, Alberto Silvio Chiappa, Alessandro Marin Vargas, Axel Bisi

Here we provide the neural data, activation and predictions for the best models and result dataframes of our article "Task-driven neural network models predict neural dynamics of proprioception". It contains the behavioral and neural experimental data (cu ...

EPFL Infoscience2024

Generalizing Bulk-Synchronous Parallel Processing for Data Science: From Data to Threads and Agent-Based Simulations

Zilu Tian

Agent-based simulations have been widely applied in many disciplines, by scientists and engineers alike. Scientists use agent-based simulations to tackle global problems, including alleviating poverty, reducing violence, and predicting the impact of pandem ...

EPFL2023

Design Space Exploration for Partitioning Dataflow Program on CPU-GPU Heterogeneous System

Marco Mattavelli, Simone Casale Brunet, Aurélien François Gilbert Bloch

Dataflow programming is a methodology that enables the development of high-level, parametric programs that are independent of the underlying platform. This approach is particularly useful for heterogeneous platforms, as it eliminates the need to rewrite ap ...

SPRINGER2023

Post-Moore's Law Fusion: High-Bandwidth Memory, Accelerators, and Native Half-Precision Processing for CPU-Local Analytics

Anastasia Ailamaki, Viktor Sanca

Modern data management systems aim to provide both cutting-edge functionality and hardware efficiency. With the advent of AI-driven data processing and the post-Moore Law era, traditional memory-bound scale-up data management operations face scalability ch ...

2023

Cluster Catalysis with Lattice Oxygen: Tracing Oxygen Transport from a Magnetite (001) Support onto Small Pt Clusters

Wolfgang Harbich, Giulia Elisabetta Pacchioni, Ke Zhang

Oxidation catalysis on reducible oxide-supported small metal clusters often involves lattice oxygen. The present work aims at differentiating whether the reaction takes place at the cluster/support interface or on the cluster. To that purpose, we trace the ...

AMER CHEMICAL SOC2021

Communication trade-offs for Local-SGD with large step size

Aymeric Daphnis Kévin Dieuleveut

Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning. However, in practice, its convergence is bottlenecked by slow communication rounds between worker nodes. A natural solution to reduce communication is to use the " ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2019

NRStitcher: non-rigid stitching of terapixel-scale volumetric images

Anne Bonnin, Arttu Ilari Miettinen

A Summary: In modern microscopy, the field of view is often increased by obtaining an image mosaic, where multiple sub-images are taken side-by-side and combined post-acquisition. Mosaic imaging often leads to long imaging times that can increase the proba ...

Oxford University Press2019

Local SGD Converges Fast and Communicates Little

Sebastian Urban Stich

Mini-batch stochastic gradient descent (SGD) is state of the art in large scale distributed training. The scheme can reach a linear speedup with respect to the number of workers, but this is rarely seen in practice as the scheme often suffers from large ne ...

2019

Decentralized clustering for node-variant graph filtering with graph diffusion LMS

Ali H. Sayed, Roula Nassif

In this work, we consider the problem of estimating the coefficients of linear shift-invariant FIR graph filters. We assume hybrid node-varying graph filters where the network is decomposed into clusters of nodes and, within each cluster, all nodes have th ...

IEEE2018

Don't cry over spilled records: Memory elasticity of data-parallel applications and its application to cluster scheduling

Willy Zwaenepoel, Syed Mohammad Aunn Raza, Calin Iorgulescu, Florin Dinu

Understanding the performance of data-parallel workloads when resource-constrained has significant practical importance but unfortunately has received only limited attention. This paper identifies, quantifies and demonstrates memory elasticity, an intrinsi ...

The USENIX Association2017

Distributed Time Series Analytics

Tian Guo

In recent years time series data has become ubiquitous thanks to affordable sensors and advances in embedded technology. Large amount of time-series data are continuously produced in a wide spectrum of applications, such as sensor networks, medical monitor ...

EPFL2017

Big-Data Streaming Applications Scheduling Based on Staged Multi-Armed Bandits

David Atienza Alonso, Karim Kanoun, Cem Tekin

Several techniques have been recently proposed to adapt Big-Data streaming applications to existing many core platforms. Among these techniques, online reinforcement learning methods have been proposed that learn how to adapt at run-time the throughput and ...

Institute of Electrical and Electronics Engineers2016

Big-Data Streaming Applications Scheduling Based on Staged Multi-armed Bandits

David Atienza Alonso, Karim Kanoun, Cem Tekin

Institute of Electrical and Electronics Engineers2016

Scalable splitting algorithms for big-data interferometric imaging in the SKA era

Jean-Philippe Thiran, Yves Wiaux, Jason Douglas McEwen, Rafael Eduardo Carrillo Rangel

In the context of next generation radio telescopes, like the Square Kilometre Array, the efficient processing of large-scale datasets is extremely important. Convex optimisation tasks under the compressive sensing framework have recently emerged and provid ...

OUP2016

SigCO: Mining Significant Correlations via a Distributed Real-time Computation Engine

Karl Aberer, Jean Paul Calbimonte Perez, Tian Guo, Hao Zhuang

The dramatic rise of time-series data produced in a variety of contexts, such as stock markets, mobile sensing, sensor networks, data centre monitoring, etc., has fuelled the development of large-scale distributed real-time computation systems (e.g., Apach ...

2015

Language Support for Distributed Functional Programming

Heather Christine Klug

Software development has taken a fundamental turn. Software today has gone from simple, closed programs running on a single machine, to massively open programs, patching together user experiences byway of responses received via hundreds of network requests ...

EPFL2015

Containers and Aggregates, Mutators and Isolates for Reactive Programming

Martin Odersky, Philipp Haller, Aleksandar Prokopec

Many programs have an inherently reactive nature imposed by the functional dependencies between their data and external events. Classically, these dependencies are dealt with using callbacks. Reactive programming with first-class reactive values is a parad ...

2014

X-Stream: Edge-centric Graph Processing using Streaming Partitions

Willy Zwaenepoel, Amitabha Roy, Ivo Mihailovic

X-Stream is a system for processing both in-memory and out-of-core graphs on a single shared-memory machine. While retaining the scatter-gather programming model with state stored in the vertices, X-Stream is novel in (i) using an edge-centric rather than ...

ACM2013

Spatiotemporal Coherence Tracking

Mohammad Alisafaee

Chip-multiprocessors require a coherence directory to track data sharing and order accesses to the shared data. Scaling coherence directories to support a large number of cores is challenging due to excessive area requirements of the directories. The state ...

2012