doi.bio/phillip_pham

Phillip Pham

Biography

Phillip Pham is a software engineer at Waymo, where he applies machine learning to motion planning. Pham previously worked in Google Research, where his research focused on increasing model capacity and understanding inductive biases of neural networks. He has also worked on internal web applications at Google. Pham holds a B.S. in Mathematics from Duke University, an M.A. in Mathematics from the University of Pennsylvania, and studied statistics at the University of Washington.

Published Works

Pham has authored several publications in the fields of machine intelligence, natural language processing, and software engineering. His notable works include:

ReadTwice: Reading Very Large Documents with Memories – Proposes ReadTwice, a simple and effective approach that combines the advantages of existing methods to modify Transformers to model long-range dependencies.
OmniNet: Omnidirectional Representations from Transformers – Introduces OmniNet, which allows each token to attend to all tokens in the entire network, providing a form of extreme attention mechanism with a full-width and depth receptive field.
Long Range Arena: A Benchmark for Efficient Transformers – Presents a systematic benchmark specifically focused on evaluating model quality in long-context scenarios, addressing the challenge of scaling Transformers to long sequence lengths.
Big Bird: Transformers for Longer Sequences – Proposes BigBird, a sparse attention mechanism that reduces quadratic dependency on sequence length, improving performance on various NLP tasks.
ETC: Encoding Long and Structured Inputs in Transformers – Introduces a new Transformer architecture, ETC, that addresses the limitations of scaling input length and ingesting structured inputs, using a global-local attention mechanism.