Hadoop Acceleration with Scalable Merging Algorithms
-- Weikuan Yu, Auburn University
Cloud computing has emerged as a new computing paradigm for the government and industry to process and analyze increasingly larger volumes of data. Hadoop implements the MapReduce programming model for cloud computing. However, it faces a number of issues to achieve the best performance from the underlying system. These include a serialization barrier that delays the reduce phase, repetitive merges and disk access, and lack of capability to leverage latest high speed interconnects. This presentation will shed light on a number of recent efforts to leverage high speed interconnects and accelerate Hadoop for fast data analytics.
Weikuan Yu is currently an Assistant Professor in the Department of Computer Science and Software Engineering at Auburn University. He directs the Parallel Architecture and System Laboratory (PASL) which hosts a NVIDIA teaching center at Auburn University and a 30Teraflop, 80-node heterogenous GPU+CPU computer cluster. Yu has research interests on cloud computing, high-performance computing, computer architecture, file and storage systems, and interdisciplinary research on climate modeling and computational biology. Yu's research is sponsored by NASA, NSF, ORNL, Mellanox, NVIDIA, and Auburn University.