doi.bio/kawin_ethayarajh

Kawin Ethayarajh

Biography

Kawin Ethayarajh is a fifth-year Ph.D. student in the Stanford AI Lab (SAIL), advised by Dan Jurafsky, where he works on behaviour-bound machine learning. He was admitted to Stanford University in Autumn 2019 and previously studied at the University of Toronto, where he was a National Scholar.

Research Focus

Ethayarajh's research focuses on the intersection of machine learning and human behaviour. He borrows from fields like economics to create algorithms, tools, and platforms that are compatible with the behaviour of real-world actors such as workers, firms, and states.

Notable Works

• Principal-Agent Problems in Data Creation: Ethayarajh explores the conflict in incentives between those who pay for data and those who produce it, and how this results in datasets being much simpler than the real-world problems they aim to reflect. He proposes frameworks to understand this discrepancy and create more accurate datasets, such as SHP, a large-scale dataset of human preferences over text.
• Understanding Dataset Difficulty with V-Usable Information: This paper, which received the ICML 2022 Outstanding Paper award, further explores the complexity of datasets and how they can be better understood and utilised.
• Pluralistic Model Alignment: Here, Ethayarajh draws connections between behavioural economics and model alignment, arguing for a more pluralistic approach to alignment. He introduces HALOs (human-aware losses), a family of losses that include the popular KTO option, used for aligning LLMs with unpaired and imbalanced human feedback.
• Model Alignment as Prospect Theoretic Optimisation: This paper was selected for ICML 2024 with a spotlight recognition, placing it in the top 3.5% of submissions.
• Cost-Sensitive Evaluation: Ethayarajh investigates the discrepancy between the ML models favoured by researchers and those deployed by firms, modelling the trade-offs and designing evaluation pipelines that better reflect real-world considerations.
• Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking: This work introduces Dynaboard, an evaluation platform used to host various challenges and collect dynamic adversarial data.
• Utility is in the Eye of the User: A Critique of NLP Leaderboards: Co-authored with Dan Jurafsky and presented at EMNLP 2020, this paper critiques the current evaluation landscape in NLP and the reliance on leaderboards.

Awards and Recognition

• ICML 2022 Outstanding Paper Award
• Facebook Fellowship
• NSERC PGS-D during Ph.D. studies
• National Scholar at the University of Toronto
• BMO (Bank of Montreal) Scholarship, valued at $75,000
• Governor General's Academic Medal
• Rhodes Scholarship Finalist
• NSERC Undergraduate Student Research Award
• 1st Place, Social Spark Case Challenge, Engineers without Borders
• Gold Medalist (Engineering), Toronto-wide Science Fair

Youtube Videos

Youtube Title: Leaderboardism in NLP: panel with Kawin Ethayarajh, Jesse Dodge, Rachael Tatman & Anna Rogers

Youtube Link: link

Youtube Channel Name: Anna Rogers

Youtube Channel Link: https://www.youtube.com/@annarogers1411

Leaderboardism in NLP: panel with Kawin Ethayarajh, Jesse Dodge, Rachael Tatman & Anna Rogers

Youtube Title: CSC258 Final Project: Tetris

Youtube Link: link

Youtube Channel Name: Kawin Ethayarajh

Youtube Channel Link: https://www.youtube.com/@kawinethayarajh9460

CSC258 Final Project: Tetris

Youtube Title: Berkeley - What Big Models and Big Data Bring to the Table - Learning How to Play with the Machines

Youtube Link: link

Youtube Channel Name: Arya McCarthy

Youtube Channel Link: https://www.youtube.com/@aryamccarthy5971

Berkeley - What Big Models and Big Data Bring to the Table - Learning How to Play with the Machines

Youtube Title: Rasa Chats: Developing a bot to help people experiencing homelessness | Podcast

Youtube Link: link

Youtube Channel Name: Rasa

Youtube Channel Link: https://www.youtube.com/@RasaHQ

Rasa Chats: Developing a bot to help people experiencing homelessness | Podcast

Youtube Title: Panel: NLP in Practice | NLP Summit

Youtube Link: link

Youtube Channel Name: John Snow Labs

Youtube Channel Link: https://www.youtube.com/@Johnsnowlabs

Panel: NLP in Practice | NLP Summit

Youtube Title: [Livestream] Why I support banning facial recognition

Youtube Link: link

Youtube Channel Name: Rachael Tatman

Youtube Channel Link: https://www.youtube.com/@rctatman

![Livestream] Why I support banning facial recognition

Youtube Title: Foundation Models in Generative Intelligence | Gen AI Fundamentals | What is foundation model

Youtube Link: link

Youtube Channel Name: Neeraj Garg

Youtube Channel Link: https://www.youtube.com/@NeerajGarg

Foundation Models in Generative Intelligence | Gen AI Fundamentals | What is foundation model

Youtube Title: Rachael Tatman | Rasa, Kaggle | Conversational AI | CTDS.Show #

Youtube Link: link

Youtube Channel Name: Chai Time Data Science

Youtube Channel Link: https://www.youtube.com/@ChaiTimeDataScience

Rachael Tatman | Rasa, Kaggle | Conversational AI | CTDS.Show #

Youtube Title: What Are Foundation Models? | On the Opportunities and Risks of Foundation Models

Youtube Link: link

Youtube Channel Name: Jordan Harrod

Youtube Channel Link: https://www.youtube.com/@JordanHarrod

What Are Foundation Models? | On the Opportunities and Risks of Foundation Models

Youtube Title: TrustML Young Scientist Seminar #29 20220902

Youtube Link: link

Youtube Channel Name: RIKEN AIP

Youtube Channel Link: https://www.youtube.com/@aipriken8732

TrustML Young Scientist Seminar #29 20220902

Youtube Title: Lecture 07: Foundation Models (FSDL 2022)

Youtube Link: link

Youtube Channel Name: The Full Stack

Youtube Channel Link: https://www.youtube.com/@TheFullStack

Lecture 07: Foundation Models (FSDL 2022)

Youtube Title: Charla Data Centric AI in Medicine

Youtube Link: link

Youtube Channel Name: Universidad Icesi

Youtube Channel Link: https://www.youtube.com/@universidadicesi

Charla Data Centric AI in Medicine

Youtube Title: Healthcare Conference 2024

Youtube Link: link

Youtube Channel Name: Nedbank

Youtube Channel Link: https://www.youtube.com/@nedbank

Healthcare Conference 2024

Youtube Title: Luke Dodge's Tandem skydive!

Youtube Link: link

Youtube Channel Name: Skydive Snohomish

Youtube Channel Link: https://www.youtube.com/@SkySno

Luke Dodge's Tandem skydive!

Youtube Title: S3E6: John H Moss Scholarship Winner and Room 2250's CS Graduating Student of the Year ft. Lana

Youtube Link: link

Youtube Channel Name: Room 2250

Youtube Channel Link: https://www.youtube.com/@Room-tz5ub

S3E6: John H Moss Scholarship Winner and Room 2250's CS Graduating Student of the Year ft. Lana

Youtube Title: SP111 Winter 2011 Brennan CCC Jesse Dodge Speech of Introduction

Youtube Link: link

Youtube Channel Name: Jessedodge1991

Youtube Channel Link: https://www.youtube.com/@Jessedodge1991

SP111 Winter 2011 Brennan CCC Jesse Dodge Speech of Introduction