Research
A paper is listed when one of its authors is affiliated with the lab. Data: OpenAlex.
DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math ImagesAllen Institute for AI · arXiv (Cornell University)1 citationJan 24, 2025s1: Simple test-time scalingAllen Institute for AI · Conference on Empirical Methods in Natural Language Processing (EMNLP)76 citationsJan 1, 2025Know Your Limits: A Survey of Abstention in Large Language ModelsAllen Institute for AI · Transactions of the Association for Computational Linguistics42 citationsJan 1, 2025The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-DeterminismAllen Institute for AI · Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL)29 citationsJan 1, 2025RewardBench: Evaluating Reward Models for Language ModelingAllen Institute for AI · Findings of the Association for Computational Linguistics: NAACL13 citationsJan 1, 2025Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with PsychometricsAllen Institute for AI · Findings of the Association for Computational Linguistics: NAACL10 citationsJan 1, 2025REL-A.I.: An Interaction-Centered Approach To Measuring Human-LM RelianceAllen Institute for AI · Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL)10 citationsJan 1, 2025Learning Syntax Without Planting Trees: Understanding Hierarchical Generalization in TransformersAllen Institute for AI, Microsoft · Transactions of the Association for Computational Linguistics10 citationsJan 1, 2025Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data GenerationAllen Institute for AI · Annual Meeting of the Association for Computational Linguistics (ACL)9 citationsJan 1, 2025Ai2 Scholar QA: Organized Literature Synthesis with AttributionAllen Institute for AI · Annual Meeting of the Association for Computational Linguistics (ACL)9 citationsJan 1, 2025Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language ModelsAllen Institute for AI · Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL)8 citationsJan 1, 2025Why and How LLMs Hallucinate: Connecting the Dots with Subsequence AssociationsAllen Institute for AI · neural information processing systems7 citationsJan 1, 2025Measuring what Matters: Construct Validity in Large Language Model BenchmarksAllen Institute for AI, Google DeepMind · neural information processing systems7 citationsJan 1, 2025OLMES: A Standard for Language Model EvaluationsAllen Institute for AI · Findings of the Association for Computational Linguistics: NAACL7 citationsJan 1, 2025Assessing Dialect Fairness and Robustness of Large Language Models in Reasoning TasksAllen Institute for AI, Microsoft · Annual Meeting of the Association for Computational Linguistics (ACL)6 citationsJan 1, 2025OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training TokensAllen Institute for AI · Annual Meeting of the Association for Computational Linguistics (ACL)6 citationsJan 1, 2025The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed TextAllen Institute for AI, Google DeepMind, Hugging Face · neural information processing systems5 citationsJan 1, 2025Re-evaluating Automatic LLM System Ranking for Alignment with Human PreferenceAllen Institute for AI · Findings of the Association for Computational Linguistics: NAACL5 citationsJan 1, 2025Generalizing Verifiable Instruction FollowingAllen Institute for AI · neural information processing systems4 citationsJan 1, 2025Wait, We Don’t Need to “Wait”! Removing Thinking Tokens Improves Reasoning EfficiencyAllen Institute for AI · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)4 citationsJan 1, 2025Contextualized Evaluations: Judging Language Model Responses to Underspecified QueriesAllen Institute for AI · Transactions of the Association for Computational Linguistics4 citationsJan 1, 2025LlamaPIE: Proactive In-Ear Conversation AssistantsAllen Institute for AI · Findings of the Association for Computational Linguistics: ACL (Findings of ACL)4 citationsJan 1, 2025MoLA: MoE LoRA with Layer-wise Expert AllocationAllen Institute for AI, Alphabet · Findings of the Association for Computational Linguistics: NAACL4 citationsJan 1, 2025Debatable Intelligence: Benchmarking LLM Judges via Debate Speech EvaluationAllen Institute for AI · Conference on Empirical Methods in Natural Language Processing (EMNLP)3 citationsJan 1, 2025SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific LiteratureAllen Institute for AI · Conference on Empirical Methods in Natural Language Processing (EMNLP)3 citationsJan 1, 2025