doi.bio/kawin_ethayarajh
Kawin Ethayarajh
Biography
Kawin Ethayarajh is a fifth-year Ph.D. student in the Stanford AI Lab (SAIL), advised by Dan Jurafsky, where he works on behaviour-bound machine learning. He was admitted to Stanford University in Autumn 2019 and previously studied at the University of Toronto, where he was a National Scholar.
Research Focus
Ethayarajh's research focuses on the intersection of machine learning and human behaviour. He borrows from fields like economics to create algorithms, tools, and platforms that are compatible with the behaviour of real-world actors such as workers, firms, and states.
Notable Works
- Principal-Agent Problems in Data Creation: Ethayarajh explores the conflict in incentives between those who pay for data and those who produce it, and how this results in datasets being much simpler than the real-world problems they aim to reflect. He proposes frameworks to understand this discrepancy and create more accurate datasets, such as SHP, a large-scale dataset of human preferences over text.
- Understanding Dataset Difficulty with V-Usable Information: This paper, which received the ICML 2022 Outstanding Paper award, further explores the complexity of datasets and how they can be better understood and utilised.
- Pluralistic Model Alignment: Here, Ethayarajh draws connections between behavioural economics and model alignment, arguing for a more pluralistic approach to alignment. He introduces HALOs (human-aware losses), a family of losses that include the popular KTO option, used for aligning LLMs with unpaired and imbalanced human feedback.
- Model Alignment as Prospect Theoretic Optimisation: This paper was selected for ICML 2024 with a spotlight recognition, placing it in the top 3.5% of submissions.
- Cost-Sensitive Evaluation: Ethayarajh investigates the discrepancy between the ML models favoured by researchers and those deployed by firms, modelling the trade-offs and designing evaluation pipelines that better reflect real-world considerations.
- Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking: This work introduces Dynaboard, an evaluation platform used to host various challenges and collect dynamic adversarial data.
- Utility is in the Eye of the User: A Critique of NLP Leaderboards: Co-authored with Dan Jurafsky and presented at EMNLP 2020, this paper critiques the current evaluation landscape in NLP and the reliance on leaderboards.
Awards and Recognition
- ICML 2022 Outstanding Paper Award
- Facebook Fellowship
- NSERC PGS-D during Ph.D. studies
- National Scholar at the University of Toronto
- BMO (Bank of Montreal) Scholarship, valued at $75,000
- Governor General's Academic Medal
- Rhodes Scholarship Finalist
- NSERC Undergraduate Student Research Award
- 1st Place, Social Spark Case Challenge, Engineers without Borders
- Gold Medalist (Engineering), Toronto-wide Science Fair
Youtube Videos
Youtube Title: Leaderboardism in NLP: panel with Kawin Ethayarajh, Jesse Dodge, Rachael Tatman & Anna Rogers
Youtube Link: link
Youtube Channel Name: Anna Rogers
Youtube Channel Link: https://www.youtube.com/@annarogers1411
Leaderboardism in NLP: panel with Kawin Ethayarajh, Jesse Dodge, Rachael Tatman & Anna Rogers
Youtube Title: CSC258 Final Project: Tetris
Youtube Link: link
Youtube Channel Name: Kawin Ethayarajh
Youtube Channel Link: https://www.youtube.com/@kawinethayarajh9460
CSC258 Final Project: Tetris
Youtube Title: Berkeley - What Big Models and Big Data Bring to the Table - Learning How to Play with the Machines
Youtube Link: link
Youtube Channel Name: Arya McCarthy
Youtube Channel Link: https://www.youtube.com/@aryamccarthy5971
Berkeley - What Big Models and Big Data Bring to the Table - Learning How to Play with the Machines
Youtube Title: Rasa Chats: Developing a bot to help people experiencing homelessness | Podcast
Youtube Link: link
Youtube Channel Name: Rasa
Youtube Channel Link: https://www.youtube.com/@RasaHQ
Rasa Chats: Developing a bot to help people experiencing homelessness | Podcast
Youtube Title: Panel: NLP in Practice | NLP Summit
Youtube Link: link
Youtube Channel Name: John Snow Labs
Youtube Channel Link: https://www.youtube.com/@Johnsnowlabs
Panel: NLP in Practice | NLP Summit
Youtube Title: [Livestream] Why I support banning facial recognition
Youtube Link: link
Youtube Channel Name: Rachael Tatman
Youtube Channel Link: https://www.youtube.com/@rctatman
![Livestream] Why I support banning facial recognition
Youtube Title: Foundation Models in Generative Intelligence | Gen AI Fundamentals | What is foundation model
Youtube Link: link
Youtube Channel Name: Neeraj Garg
Youtube Channel Link: https://www.youtube.com/@NeerajGarg
Foundation Models in Generative Intelligence | Gen AI Fundamentals | What is foundation model
Youtube Title: Rachael Tatman | Rasa, Kaggle | Conversational AI | CTDS.Show #
Youtube Link: link
Youtube Channel Name: Chai Time Data Science
Youtube Channel Link: https://www.youtube.com/@ChaiTimeDataScience
Rachael Tatman | Rasa, Kaggle | Conversational AI | CTDS.Show #
Youtube Title: What Are Foundation Models? | On the Opportunities and Risks of Foundation Models
Youtube Link: link
Youtube Channel Name: Jordan Harrod
Youtube Channel Link: https://www.youtube.com/@JordanHarrod
What Are Foundation Models? | On the Opportunities and Risks of Foundation Models
Youtube Title: TrustML Young Scientist Seminar #29 20220902
Youtube Link: link
Youtube Channel Name: RIKEN AIP
Youtube Channel Link: https://www.youtube.com/@aipriken8732
TrustML Young Scientist Seminar #29 20220902
Youtube Title: Lecture 07: Foundation Models (FSDL 2022)
Youtube Link: link
Youtube Channel Name: The Full Stack
Youtube Channel Link: https://www.youtube.com/@TheFullStack
Lecture 07: Foundation Models (FSDL 2022)
Youtube Title: Charla Data Centric AI in Medicine
Youtube Link: link
Youtube Channel Name: Universidad Icesi
Youtube Channel Link: https://www.youtube.com/@universidadicesi
Charla Data Centric AI in Medicine
Youtube Title: Healthcare Conference 2024
Youtube Link: link
Youtube Channel Name: Nedbank
Youtube Channel Link: https://www.youtube.com/@nedbank
Healthcare Conference 2024
Youtube Title: Luke Dodge's Tandem skydive!
Youtube Link: link
Youtube Channel Name: Skydive Snohomish
Youtube Channel Link: https://www.youtube.com/@SkySno
Luke Dodge's Tandem skydive!
Youtube Title: S3E6: John H Moss Scholarship Winner and Room 2250's CS Graduating Student of the Year ft. Lana
Youtube Link: link
Youtube Channel Name: Room 2250
Youtube Channel Link: https://www.youtube.com/@Room-tz5ub
S3E6: John H Moss Scholarship Winner and Room 2250's CS Graduating Student of the Year ft. Lana
Youtube Title: SP111 Winter 2011 Brennan CCC Jesse Dodge Speech of Introduction
Youtube Link: link
Youtube Channel Name: Jessedodge1991
Youtube Channel Link: https://www.youtube.com/@Jessedodge1991
SP111 Winter 2011 Brennan CCC Jesse Dodge Speech of Introduction