doi.bio/niklas_muennighoff

Niklas Muennighoff

Niklas Muennighoff is a researcher in Large Language Models (LLMs) at Contextual AI. He will be starting a PhD at Stanford in September. Muennighoff completed his bachelor's degree at Peking University.

Muennighoff's research focuses on improving the usefulness of LLMs. He has worked on projects in Chinese, English, Japanese, French, and German. His work includes:

Scaling up LLMs
Improving instruction-following in models
Working on the largest text embedding benchmark (MTEB)
Building multimodal models

Publications

Muennighoff has published several papers in the field of Natural Language Processing (NLP) and LLMs. Some of his notable publications include:

MTEB: Massive Text Embedding Benchmark: This paper introduces the Massive Text Embedding Benchmark (MTEB), which aims to address the lack of proper evaluation of text embeddings. MTEB spans 8 embedding tasks, 58 datasets, and 112 languages. The paper establishes the most comprehensive benchmark of text embeddings and finds that no single text embedding method dominates across all tasks.
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting: The paper focuses on adapting the BLOOM language model to new languages not seen during its pretraining. It finds that language adaptation is effective in improving performance in new languages and that adapter-based finetuning is more effective than continued pretraining for large models.
Crosslingual Generalization through Multitask Finetuning: This work applies multitask prompted finetuning (MTF) to multilingual language models, finding that finetuning on English tasks and prompts allows for generalization to non-English languages. The paper also introduces xP3, a composite of supervised datasets in 46 languages.
FinGPT: Large Generative Models for a Small Language: The paper studies the challenges of creating LLMs for Finnish, a low-resource language. It introduces the FinGPT and BLUUMI models and evaluates them using FIN-bench, a Finnish version of BIG-bench.
What Language Model to Train if You Have One Million GPU Hours?: This work focuses on identifying the best architecture and training setup for the BLOOM language model within a given computational budget. It performs an ablation study and investigates the impact of different pre-training corpora and multilingual models.

Other Projects

In addition to his research and publications, Muennighoff has also worked on several other projects, including:

OctoPack: Instruction tuning code for large language models.
SGPT: GPT sentence embeddings for semantic search.
GRITLM: Generative Representational Instruction Tuning.
Bloomprint: A collection of 140 simple rules for living a long life, based on research papers.

Youtube Videos

Youtube Title: Speech of Niklas Muennighoff, Global BBA emblematic student | ESSEC Commencement Day 2022

Youtube Link: link

Youtube Channel Name: ESSEC Business School

Youtube Channel Link: https://www.youtube.com/@essec

Speech of Niklas Muennighoff, Global BBA emblematic student | ESSEC Commencement Day 2022

Youtube Title: Cohere For AI - Community Talks: Niklas Muennighoff

Youtube Link: link

Youtube Channel Name: Cohere

Youtube Channel Link: https://www.youtube.com/@CohereAI

Cohere For AI - Community Talks: Niklas Muennighoff

Youtube Title: C4AI @ NeurIPS, 2023 - Niklas Muennighoff on Scaling Data-Constrained Language Models

Youtube Link: link

Youtube Channel Name: Cohere

Youtube Channel Link: https://www.youtube.com/@CohereAI

C4AI @ NeurIPS, 2023 - Niklas Muennighoff on Scaling Data-Constrained Language Models

Youtube Title: Scaling Data-Constrained Language Models | Talk at IST-Unbabel Seminar

Youtube Link: link

Youtube Channel Name: Niklas Muennighoff

Youtube Channel Link: https://www.youtube.com/@Muennighoff

Scaling Data-Constrained Language Models | Talk at IST-Unbabel Seminar

Youtube Title: Niklas Muennighoff - From GPU poor to poor GPU rich

Youtube Link: link

Youtube Channel Name: Sasha Rush 🤗

Youtube Channel Link: https://www.youtube.com/@srush_nlp

Niklas Muennighoff - From GPU poor to poor GPU rich

Youtube Title: MTEB: Massive Text Embedding Benchmark | Video Summary

Youtube Link: link

Youtube Channel Name: Niklas Muennighoff

Youtube Channel Link: https://www.youtube.com/@Muennighoff

MTEB: Massive Text Embedding Benchmark | Video Summary

Youtube Title: Crosslingual Generalization through Multitask Finetuning | Video Summary

Youtube Link: link

Youtube Channel Name: Niklas Muennighoff

Youtube Channel Link: https://www.youtube.com/@Muennighoff

Crosslingual Generalization through Multitask Finetuning | Video Summary

Youtube Title: Graduation Speech, Peking University

Youtube Link: link

Youtube Channel Name: Niklas Muennighoff

Youtube Channel Link: https://www.youtube.com/@Muennighoff

Graduation Speech, Peking University

Youtube Title: Generative Representational Instruction Tuning and Agents for Video Creation | Multimodal Weekly 42

Youtube Link: link

Youtube Channel Name: Twelve Labs

Youtube Channel Link: https://www.youtube.com/@Twelve_Labs

Generative Representational Instruction Tuning and Agents for Video Creation | Multimodal Weekly 42

Youtube Title: Crosslingual Generalization through Multitask Finetuning (BLOOMZ & mT0)

Youtube Link: link

Youtube Channel Name: Samuel Albanie

Youtube Channel Link: https://www.youtube.com/@SamuelAlbanie1

Crosslingual Generalization through Multitask Finetuning (BLOOMZ & mT0)

Youtube Title: Generative Representational Instruction Tuning Explained

Youtube Link: link

Youtube Channel Name: Unify

Youtube Channel Link: https://www.youtube.com/@unifyai

Generative Representational Instruction Tuning Explained

Youtube Title: DeepPavlov Community Call #11

Youtube Link: link

Youtube Channel Name: DeepPavlov

Youtube Channel Link: https://www.youtube.com/@DeepPavlov

DeepPavlov Community Call #11

Youtube Title: Forget models, build a data flywheel | Machine Learning Monthly January 2021

Youtube Link: link

Youtube Channel Name: Daniel Bourke

Youtube Channel Link: https://www.youtube.com/@mrdbourke

Forget models, build a data flywheel | Machine Learning Monthly January 2021

sness@sness.net