王林楠 (Linnan Wang)

Office 351, CIT
Department of Computer Science
Brown University
Providence, RI 02906

Google Scholar |  GitHub |  LinkedIn

My Mission:

Artificial Intelligence (AI) is going to be the extension of our brains, in the same way as cars are the extension of our legs. It has already been an indispensable part of our life. Every day, AI navigates us to places, answers our queries, and recommends restaurants and movies. Overall, it amplifies what we do, augmenting our memory, giving you instant knowledge, allowing us to concentrate on doing things that are properly human.

However, designing new AI models is still reserved for experts; and the goal of my research is to democratize AI, making it accessible to everybody, such that any person regardless of their prior experiences, and any company regardless of size can deploy sophisticated AI solutions with only a few simple clicks.

I'm a senior deep learning engineer at NVIDIA. I got my Ph.D. from the CS department of Brown University, advised by Prof.Rodrigo Fonseca. Before Brown, I was a OMSCS student at Gatech while being a full time software developer at Dow Jones. I acquired my bachelor degree from University of Electronic Science and Technology of China (UESTC) at the beautiful Qing Shui He campus in 2011. I have been incredibly fortunate to work closely with Yiyang Zhao, Junyu Zhang, Yuandong Tian, Saining Xie, Yi Yang, Wei Wu, George Bosilca, and 2022 Turing Fellow Jack Dongarra.

Publications:

Wang, Linnan, Chen-han Yu, Satish Salian, Slawomir Kierat, Szymon Migacz, Alex Fit Florea
GPUNet: Searching the Deployable Convolution Neural Networks for GPUs
CVPR-2022 ·  Acceptance Rate: 25.3% · IEEE Conference on Computer Vision and Pattern Recognition
Paper · Code

Zhao, Yiyang, Linnan Wang, Kevin Yang, Tianjun Zhang, Tian Guo, Yuandong Tian
Multi-objective Optimization by Learning Space Partitions
ICLR-2022 ·  Acceptance Rate: 32% · International Conference on Learning Representations
Paper 
I just love this work, the best extension to LA-MCTS so far.

Zhao, Yiyang, Linnan Wang, Yuandong Tian, Rodrigo Fonseca, Tian Guo
Neural Architecture Search via Multi-Objective Optimization
Paper to appear 

Yang, Kevin, Tianjun Zhang, Chris Cummins, Brandon Cui, Benoit Steiner, Linnan Wang, Joseph E. Gonzalez, Dan Klein, Yuandong Tian
Learning Space Partitions for Path Planning
NeurIPS-2021 ·  Acceptance Rate: 26% · Advances in Neural Information Processing Systems
Paper

Works at Brown:

Linnan Wang
Building an Intelligent Agent to Design Neural Networks
Ph.D Thesis 

Zhao, Yiyang, Linnan Wang (equally contributed), Yuandong Tian, Rodrigo Fonseca, Tian Guo
ICML-2021 · Acceptance Rate: 3% · International Conference on Machine Learning
Few-shot Neural Architecture Search
Paper ·  Code ·  FB AI Blog
Long Oral

Wang, Linnan, Saining Xie, Teng Li, Rodrigo Fonseca, Yuandong Tian
Sample-Efficient Neural Architecture Search by Learning Action Space for Monte Carlo Tree Search
TPAMI-2021 · IEEE Transactions on Pattern Analysis and Machine Intelligence
Paper ·  FB Internal Pitch ·  Code 
Our agent designs a neural network that reaches 99% top-1 accuracy on CIFAR-10.

Wang, Linnan, Rodrigo Fonseca, Yuandong Tian
Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search
NeurIPS-2020 ·  Acceptance Rate: 20% · Advances in Neural Information Processing Systems
Paper ·  Poster ·  Code 
LA-MCTS is used by the 3rd (JetBrains) and 8th place (KAIST) teams in the NeurIPS black-box optimization challenge.

Wang, Linnan, Yiyang Zhao, Yuu Jinnai, Yuandong Tian, Rodrigo Fonseca
Neural Architecture Search using Deep Neural Networks and Monte Carlo Tree Search
AAAI-2020 ·  Acceptance Rate: 16% ·  AAAI conference on Artificial Intelligence
Paper ·  Poster ·  Code

Wang, Linnan, Wei Wu, Junyu Zhang, Hang Liu, George Bosilca, Maurice Herlihy, Rodrigo Fonseca
SuperNeurons: FFT-based Gradient Sparsification in the Distributed Training of Deep Neural Networks
HPDC-2020  ·  Acceptance Rate: 18%  ·  ACM Symposium on High-Performance Parallel and Distributed Computing
Paper ·  Talk ·  Code

Wang, Linnan, Jinmian Ye, Yiyang Zhao, Wei Wu, Ang Li, Shuaiwen Leon Song, Zenglin Xu, Tim Kraska
SuperNeurons: Dynamic GPU Memory Management for Training Deep Nonlinear Neural Networks
PPoPP-2018 ·  Acceptance Rate: 21%  ·  ACM Symposium on Principles and Practice of Parallel Programming
Paper ·  Talk ·  Presentation  ·  Code  

Wang, Linnan, Yi Yang, Renqiang Min, and Srimat Chakradhar
Accelerating Deep Neural Network Training with Inconsistent Stochastic Gradient Descent
Neural Networks-2017
Paper ·  Patent

Wang, Linnan, Wei Wu, Zenglin Xu, Jianxiong Xiao, and Yi Yang
BLASX: A High Performance Level-3 BLAS Library for Heterogeneous MultiGPU Computing
ICS-2016 · Acceptance Rate: 24% ·  International Conference on Supercomputing
Paper ·  Poster ·  Code ·  Presentation 

Ye, Jinmian, Linnan Wang, Guangxi Li, Di Chen, Shandian Zhe, Xinqi Chu, Zenglin Xu
Learning Compact Recurrent Neural Networks with Block-Term Tensor Decomposition
CVPR-2018 ·  Acceptance Rate: 29.6% · IEEE Conference on Computer Vision and Pattern Recognition
Paper ·  Poster 

Luo, Xi , Wei Wu, George Bosilca, Thananon Patinyasakdikul, Linnan Wang, Jack Dongarra
ADAPT: An Event-based Adaptive Collective Communication Framework
HPDC-2018 · Acceptance Rate: 20% · ACM Symposium on High-Performance Parallel and Distributed Computing
Paper · 

Li, Ang, Weifeng Liu, Linnan Wang, Kevin Barker, Shuaiwen Leon Song
Warp-Consolidation: A Novel Execution Model for Modern GPUs
ICS-2018 · Acceptance Rate: 26% · International Conference on Supercomputing
Paper

Zhao, Yiyang, Linnan Wang, Wei Wu, George Bosilca, Richard Vuduc, Jinmian Ye, Wenqi Tang, and Zenglin Xu.
Efficient Communications in Training Large Scale Neural Networks
MM-2017, workshop in ACM international conference on Multimedia
Paper

Projects:

Automated Machine Learning: building an AI that builds AI.

  • AlphaX: inspired by AlphaGo, we build the very first NAS search algorithm based on Monte Carlo Tree Search (MCTS). We showed Neural Networks designed by AlphaX improve the downstream applications such as detection, style transfer, image captioning, and many others.

    • Github: https://github.com/linnanwang/AlphaX-NASBench101

  • LA-MCTS: we find that different action space used in MCTS significantly affects the search efficiency, which motivates the idea of learning action space for MCTS (LA-MCTS). This project contains 1) a distributed version that is scalable to hundreds of GPUs to push SoTA results, and 2) a one-shot version that let you get a working result within a few GPU days. You can find the entire pipeline (search and training) for doing NAS here.

    With LA-MCTS, we have achieved SoTA results on many CV tasks including CIFAR-10, ImageNet and detection. Besides, LA-MCTS also achieves strong performance in general black-box optimization and reinforcement learning benchmarks, in particular for high-dimensional problems.

    • Github: https://github.com/facebookresearch/LaMCTS

    Machine Learning System: building efficient distributed systems for AI.

  • SuperNeurons: this project builds a C++ Deep Learning framework, which features a dynamic GPU memory scheduling run-time to enable the neural network training far beyond the GPU DRAM capacity, and a FFT based gradient compression protocol for the efficient distributed DNN training.

    • Github: https://github.com/linnanwang/superneurons-release

  • BLASX: this project builds a level-3 BLAS library for heterogeneous multiGPUs. Due to the novel tile-cache design to avoid unnecessarily data-swapping, BLASX is 30% faster than commercial cuBLAS-XT from NVIDIA.

    • Github: https://github.com/linnanwang/BLASX

    Patent:

    Awards:

    Academic Services:

    Professional Experiences: