Publications

Ofir Arviv, Miruna Clinciu, Kaustubh Dhole, Rotem Dror, Sebastian Gehrmann, Eliya Habba, Itay Itzhak, Simon Mille, Yotam Perlitz, Enrico Santus, João Sedoc, Michal Shmueli Scheuer, Gabriel Stanovsky, Oyvind Tafjord (Editors)

Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs COLM

🎉 Spotlight Presentation
Itay Itzhak, Yonatan Belinkov, Gabriel Stanovsky

Looking Beyond The Top-1: Transformers Determine Top Tokens In Order ICML

Daria Lioubashevski, Tomer Schlank, Gabriel Stanovsky, Ariel Goldstein

Can LLMs Help Uncover Insights about LLMs? A Large-Scale, Evolving Literature Analysis of Frontier LLMs ACL

Jungsoo Park, Junmo Kang, Gabriel Stanovsky, Alan Ritter

DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation ACL Findings

Eliya Habba, Ofir Arviv, Itay Itzhak, Yotam Perlitz, Elron Bandel, Leshem Choshen, Michal Shmueli-Scheuer, Gabriel Stanovsky

PROJECT

The State and Fate of Summarization Datasets: A Survey NAACL

Noam Dahan, Gabriel Stanovsky

CODE

In-Context Learning on a Budget: A Case Study in Token Classification Workshop on Insights from Negative Results in NLP

Uri Berger, Tal Baumel, Gabriel Stanovsky

2024

Applying Intrinsic Debiasing on Downstream Tasks: Challenges and Considerations for Machine Translation EMNLP

Bar Iluz, Yanai Elazar, Asaf Yehudai, Gabriel Stanovsky

Schema-Driven Information Extraction from Heterogeneous Tables EMNLP Findings

Fan Bai, Junmo Kang, Gabriel Stanovsky, Dayne Freitag, Mark Dredze, Alan Ritter

State of What Art? A Call for Multi-Prompt LLM Evaluation TACL (@ ACL)

Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, Gabriel Stanovsky

CODE

Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias TACL (@ ACL)

Itay Itzhak, Gabriel Stanovsky, Nir Rosenfeld, Yonatan Belinkov

CODE

Leveraging Collection-Wide Similarities for Unsupervised Document Structure Extraction ACL Findings

Gili Lior, Yoav Goldberg, Gabriel Stanovsky

CODE

K-QA: A Real-World Medical Q&A Benchmark BioNLP Workshop

Itay Manes, Naama Ronn, David Cohen, Ran Ilan Ber, Zehavi Horowitz-Kugler, Gabriel Stanovsky

CODE

Navigating the Modern Evaluation Landscape: Considerations in Benchmarks and Frameworks for Large Language Models Tutorial at LREC-COLING

Leshem Choshen, Ariel Gera, Yotam Perlitz, Michal Shmueli-Scheuer, Gabriel Stanovsky

SLIDES BIB

Do Zombies Understand? A Choose-Your-Own-Adventure Exploration of Machine Cognition ACL Findings

Ariel Goldstein, Gabriel Stanovsky

POSTER

2023

WHOOPS! A Vision-and-Language Commonsense Benchmark of Heterogeneous Objects and Situations ICCV

Nitzan Bitton-Guetta*, Yonatan Bitton*, Jack Hessel, Ludwig Schmidt, Yuval Elovici, Gabriel Stanovsky, Roy Schwartz

PROJECT

Evaluating and Improving the Coreference Capabilities of Machine Translation Models EACL

Asaf Yehudai, Arie Cattan, Omri Abend, Gabriel Stanovsky

Comparing Humans and Models on a Similar Scale: Towards Cognitive Gender Bias Evaluation in Coreference Resolution CogSci

Gili Lior and Gabriel Stanovsky

CODE

Exploring the Impact of Training Data Distribution and Subword Tokenization on Gender Bias in Machine Translation AACL

Bar Iluz, Tomasz Limisiewicz, Gabriel Stanovsky, David Mareček

CODE

VASR: Visual Analogies of Situation Recognition AAAI

Yonatan Bitton, Ron Yosef, Eli Strugo, Dafna Shahaf, Roy Schwartz, Gabriel Stanovsky

You Can Have Your Data and Balance It Too: Towards Balanced and Efficient Multilingual Models SIGTYP

Tomasz Limisiewicz, Dan Malkin and Gabriel Stanovsky

The Perfect Victim: Computational Analysis of Judicial Attitudes towards Victims of Sexual Violence ICAIL

Eliya Habba, Renana Keydar, Dan Bareket, and Gabriel Stanovsky

A Large-Scale Multilingual Study of Visual Constraints on Linguistic Selection of Descriptions EACL Findings

Uri Berger, Lea Frermann, Gabriel Stanovsky, Omri Abend

Are Layout-Infused Language Models Robust to Layout Distribution Shifts? A Case Study with Scientific Documents ACL Findings

Catherine Chen, Zejiang Shen, Dan Klein, Gabriel Stanovsky, Doug Downey, Kyle Lo

2022

Winogavil: Gamified association benchmark to challenge vision-and-language models NeurIPS Datasets and Benchmarks

🎉 Selected as Featured Presentation
Yonatan Bitton, Nitzan Bitton Guetta, Ron Yosef, Yuval Elovici, Mohit Bansal, Gabriel Stanovsky, Roy Schwartz

Covid vaccine is against Covid but Oxford vaccine is made at Oxford! Semantic Interpretation of Proper Noun Compounds EMNLP

Keshav Kolluru, Gabriel Stanovsky, Mausam

GENIE: A Leaderboard for Human-in-the-Loop Evaluation of Text Generation EMNLP

Daniel Khashabi, Gabriel Stanovsky, Jonathan Bragg, Nicholas Lourie, Jungo Kasai, Yejin Choi, Noah A. Smith, Daniel S. Weld
Media coverage: VentureBeat

PROJECT

A Computational Acquisition Model for Multimodal Word Categorization NAACL

Uri Berger, Gabriel Stanovsky, Omri Abend, Lea Frermann

A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bank NAACL

🎉 Won an Outstanding Paper Award
Dan Malkin, Tomasz Limisiewicz, Gabriel Stanovsky

On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations NAACL Findings

Roy Schwartz, Gabriel Stanovsky

2021

Automated Extraction of Sentencing Decisions from Court Cases in the Hebrew Language NLLP

Mohr Wenger, Tom Kalir, Noga Berger, Carmit Chalamish, Renana Keydar, Gabriel Stanovsky

Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach EMNLP

Koren Lazar, Benny Saret, Asaf Yehudai, Wayne Horowitz, Nathan Wasserman, Gabriel Stanovsky
Media coverage: Daily Mail, NewScientist

CODE

Data Efficient Masked Language Modeling for Vision and Language EMNLP Findings

Yonatan Bitton, Gabriel Stanovsky, Michael Elhadad, Roy Schwartz

CODE

Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation EMNLP Findings

Shahar Levy, Koren Lazar, Gabriel Stanovsky

CODE

Realistic Evaluation Principles for Cross-document Coreference Resolution STARSEM

Arie Cattan, Alon Eirew, Gabriel Stanovsky, Mandar Joshi and Ido Dagan

CODE

Cross-document Coreference Resolution over Predicted Mentions ACL Findings

Arie Cattan, Alon Eirew, Gabriel Stanovsky, Mandar Joshi and Ido Dagan

CODE

Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQA NAACL

Yonatan Bitton, Gabriel Stanovsky, Roy Schwartz, Michael Elhadad

CODE

Process-Level Representation of Scientific Protocols with Interactive Annotation EACL

Ronen Tamari, Fan Bai, Alan Ritter, Gabriel Stanovsky

PROJECT

Gender Trends in Computer Science Authorship Communications of the ACM
Media coverage: New York Times, Science, AXIOS, GeekWire

Lucy Lu Wang, Gabriel Stanovsky, Luca Weihs, Oren Etzioni

BIB

2020

Gender Coreference and Bias Evaluation at WMT Conference on Machine Translation (WMT20)

Tom Kocmi, Tomasz Limisiewicz, Gabriel Stanovsky

CODE

MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metrics EMNLP

Anthony Chen, Gabriel Stanovsky, Sameer Singh, Matt Gardner

PROJECT-PAGE BIB

The Right Tool for the Job: Matching Model and Instance Complexities ACL

Roy Schwartz, Gabriel Stanovsky, Swabha Swayamdipta, Jesse Dodge, Noah Smith

CODE TALK BIB

Crowdsourcing a High-Quality Gold Standard for QA-SRL ACL

Paul Roit, Ayal Klein, Daniela Stepanov, Jonathan Mamou, Julian Michael, Gabriel Stanovsky, Luke Zettlemoyer, Ido Dagan

TALK BIB

Active Learning for Coreference Resolution using Discrete Annotation ACL

Belinda Li, Gabriel Stanovsky, Luke Zettlemoyer

Annotation-interface CODE TALK BIB

Ecological Semantics: Programming Environments for Situated Language Understanding BAICS workshop at ICLR2020

Ronen Tamari, Gabriel Stanovsky, Dafna Shahaf, Reut Tsarfaty

DEMO TALK BIB

2019

Evaluating Gender Bias in Machine Translation ACL

🎉 Nominated for Best Paper award (top 1% of submissions)
Gabriel Stanovsky, Noah Smith, Luke Zettlemoyer

BLOG CODE TALK SLIDES POSTER BIB

SemEval-2019 Task 10 - Math Question Answering SemEval

Mark Hopkins, Ronan Le Bras, Cristian Petrescu-Prahova, Gabriel Stanovsky, Hannaneh Hajishirzi, Rik Koncel-Kedziorski

CodaLab BIB

DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs NAACL

Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh and Matt Gardner

PROJECT BIB

Y’all should read this! Identifying Plurality in Second-Person Personal Pronouns in English Texts Workshop on Noisy User-generated Text (W-NUT), EMNLP workshop

Gabriel Stanovsky, Ronen Tamari

DATA POSTER BIB

Evaluating Question Answering Evaluation Machine Reading for Question Answering (MRQA), EMNLP workshop

🎉 Won the Best Paper award
Anthony Chen, Gabriel Stanovsky, Sameer Singh and Matt Gardner

BIB

On the Limits of Learning to Actively Learn Semantic Representations CoNLL

🎉 Honorable mention for best paper
Omri Koshorek, Gabriel Stanovsky, Yichu Zhou, Vivek Srikumar and Jonathan Berant

BIB

2018

Spot the Odd Man Out; Exploring the Associative Power of Lexical Resources EMNLP

Gabriel Stanovsky, Mark Hopkins

CODE POSTER BIB

Semantics as a Foreign Language EMNLP

Gabriel Stanovsky, Ido Dagan

TALK SLIDES BIB

Supervised Open Information Extraction NAACL

Gabriel Stanovsky, Julian Michael, Luke Zettlemoyer, Ido Dagan

DEMO CODE POSTER BIB

Crowdsourcing Question-Answer Meaning Representations NAACL

Julian Michael, Gabriel Stanovsky, Luheng He, Ido Dagan, Luke Zettlemoyer

DATA CODE BIB

2017

Integrating Deep Linguistic Features in Factuality Prediction over Unified Datasets ACL

Gabriel Stanovsky, Judith Eckle-Kohler, Yevgeniy Puzikov, Ido Dagan and Iryna Gurevych

DEMO CODE SLIDES POSTER BIB

Acquiring Predicate Paraphrases from News Tweets STARSEM

Vered Shwartz, Gabriel Stanovsky, and Ido Dagan

CODE BIB

A Consolidated Open Knowledge Representation for Multiple Texts EACL, LSDSem Workshop

Rachel Wities, Vered Shwartz, Gabriel Stanovsky, Meni Adler, Ori Shapira, Shyam Upadhyay, Dan Roth, Eugenio Martinez Camara, Iryna Gurevych, Ido Dagan

CODE SLIDES POSTER BIB

Recognizing Mentions of Adverse Drug Reaction in Social Media Using Knowledge-Infused Recurrent Models EACL

Gabriel Stanovsky, Daniel Gruhl, Pablo N. Mendes

DATA SLIDES POSTER BIB

Getting More Out Of Syntax with PropS arXiv preprint, March

Gabriel Stanovsky, Jessica Ficler, Ido Dagan, Yoav Goldberg

CODE SLIDES BIB

2015

Open IE as an Intermediate Structure for Semantic Tasks ACL

Gabriel Stanovsky, Ido Dagan, Mausam

OIE-EMBEDDINGS SLIDES BIB

2014

Proposition Knowledge Graphs COLING, AHA Workshop

Gabriel Stanovsky, Omer Levy, Ido Dagan

SLIDES POSTER BIB

Intermediary Semantic Representation Through Proposition Structures ACL, Semantic Parsing Workshop

Gabriel Stanovsky, Jessica Ficler, Ido Dagan, Yoav Goldberg

POSTER BIB

2026

ScheMatiQ: From Research Question to Structured Data through Interactive Schema Discovery ACL Demo

Leveraging Digitized Newspapers to Collect Summarization Data in Low-Resource Languages EACL Findings

2025

Cooking Up Creativity: A Cognitively-Inspired Approach for Enhancing LLM Creativity through Structured Representations TACL (@ EMNLP)

Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysis TACL (@ EMNLP)

More Documents, Same Length: Isolating the Challenge of Multiple Documents in RAG EMNLP Findings

Time to Talk: LLM Agents for Asynchronous Group Communication in Mafia Games EMNLP Findings

Trust Me, I'm Wrong: High-Certainty Hallucinations in LLMs EMNLP Findings

ReliableEval: A Recipe for Stochastic LLM Evaluation via Method of Moments EMNLP Findings

PromptSuite: A Task-Agnostic Framework for Multi-Prompt Generation EMNLP Demo

Proceedings of the GEM^2 Workshop: Generation Evaluation & Metrics ACL Workshop

Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs COLM

Looking Beyond The Top-1: Transformers Determine Top Tokens In Order ICML

Can LLMs Help Uncover Insights about LLMs? A Large-Scale, Evolving Literature Analysis of Frontier LLMs ACL

DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation ACL Findings

The State and Fate of Summarization Datasets: A Survey NAACL

In-Context Learning on a Budget: A Case Study in Token Classification Workshop on Insights from Negative Results in NLP

2024

Applying Intrinsic Debiasing on Downstream Tasks: Challenges and Considerations for Machine Translation EMNLP

Schema-Driven Information Extraction from Heterogeneous Tables EMNLP Findings

State of What Art? A Call for Multi-Prompt LLM Evaluation TACL (@ ACL)

Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias TACL (@ ACL)

Leveraging Collection-Wide Similarities for Unsupervised Document Structure Extraction ACL Findings

K-QA: A Real-World Medical Q&A Benchmark BioNLP Workshop

Navigating the Modern Evaluation Landscape: Considerations in Benchmarks and Frameworks for Large Language Models Tutorial at LREC-COLING

Do Zombies Understand? A Choose-Your-Own-Adventure Exploration of Machine Cognition ACL Findings

2023

WHOOPS! A Vision-and-Language Commonsense Benchmark of Heterogeneous Objects and Situations ICCV

Evaluating and Improving the Coreference Capabilities of Machine Translation Models EACL

Comparing Humans and Models on a Similar Scale: Towards Cognitive Gender Bias Evaluation in Coreference Resolution CogSci

Exploring the Impact of Training Data Distribution and Subword Tokenization on Gender Bias in Machine Translation AACL

VASR: Visual Analogies of Situation Recognition AAAI

You Can Have Your Data and Balance It Too: Towards Balanced and Efficient Multilingual Models SIGTYP

The Perfect Victim: Computational Analysis of Judicial Attitudes towards Victims of Sexual Violence ICAIL

A Large-Scale Multilingual Study of Visual Constraints on Linguistic Selection of Descriptions EACL Findings

Are Layout-Infused Language Models Robust to Layout Distribution Shifts? A Case Study with Scientific Documents ACL Findings

2022

Winogavil: Gamified association benchmark to challenge vision-and-language models NeurIPS Datasets and Benchmarks

Covid vaccine is against Covid but Oxford vaccine is made at Oxford! Semantic Interpretation of Proper Noun Compounds EMNLP

GENIE: A Leaderboard for Human-in-the-Loop Evaluation of Text Generation EMNLP

A Computational Acquisition Model for Multimodal Word Categorization NAACL

A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bank NAACL

On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations NAACL Findings

2021

Automated Extraction of Sentencing Decisions from Court Cases in the Hebrew Language NLLP

Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach EMNLP

Data Efficient Masked Language Modeling for Vision and Language EMNLP Findings

Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation EMNLP Findings

Realistic Evaluation Principles for Cross-document Coreference Resolution STARSEM

Cross-document Coreference Resolution over Predicted Mentions ACL Findings

Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQA NAACL

Process-Level Representation of Scientific Protocols with Interactive Annotation EACL

Gender Trends in Computer Science Authorship Communications of the ACM Media coverage: New York Times, Science, AXIOS, GeekWire

2020

Gender Coreference and Bias Evaluation at WMT Conference on Machine Translation (WMT20)

MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metrics EMNLP

The Right Tool for the Job: Matching Model and Instance Complexities ACL

Crowdsourcing a High-Quality Gold Standard for QA-SRL ACL

Active Learning for Coreference Resolution using Discrete Annotation ACL

Ecological Semantics: Programming Environments for Situated Language Understanding BAICS workshop at ICLR2020

2019

Evaluating Gender Bias in Machine Translation ACL

SemEval-2019 Task 10 - Math Question Answering SemEval

DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs NAACL

Y’all should read this! Identifying Plurality in Second-Person Personal Pronouns in English Texts Workshop on Noisy User-generated Text (W-NUT), EMNLP workshop

Evaluating Question Answering Evaluation Machine Reading for Question Answering (MRQA), EMNLP workshop

On the Limits of Learning to Actively Learn Semantic Representations CoNLL

2018

Spot the Odd Man Out; Exploring the Associative Power of Lexical Resources EMNLP

Semantics as a Foreign Language EMNLP

Supervised Open Information Extraction NAACL

Crowdsourcing Question-Answer Meaning Representations NAACL

2017

Integrating Deep Linguistic Features in Factuality Prediction over Unified Datasets ACL

Acquiring Predicate Paraphrases from News Tweets STARSEM

A Consolidated Open Knowledge Representation for Multiple Texts EACL, LSDSem Workshop

Recognizing Mentions of Adverse Drug Reaction in Social Media Using Knowledge-Infused Recurrent Models EACL

2016

Modeling Extractive Sentence Intersection via Subtree Entailment COLING

Gender Trends in Computer Science Authorship Communications of the ACM
Media coverage: New York Times, Science, AXIOS, GeekWire