Benchmarks
GenAI benchmarking study headed by John Craske
Provider
CMS and LITIG
Dataset name
Dataset size
TBD testcases
Benchmark type
Upcoming
Date published
February 19, 2025
Industry
Legal
The LinksAI English law benchmark
Provider
Linklaters
Dataset name
Dataset size
50 testcases
Benchmark type
Open-ended task
Date published
February 10, 2025
Industry
Legal
Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
Provider
Stanford RegLab
Dataset name
Dataset size
5000 testcases
Benchmark type
Open-ended task
Date published
June 21, 2024
Industry
Legal
Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools
Provider
Stanford
Dataset name
Dataset size
200+ testcases
Benchmark type
Open-ended RAG
Date published
June 6, 2024
Industry
Legal
Long-context retrieval benchmark on legal documents
Version of our internal dataset for evaluating large language models (LLMs) and model systems on complex legal tasks
Provider
Harvey
Dataset name
Dataset size
50+ testcases
Benchmark type
Open-ended task
Date published
August 29, 2024
Industry
Legal
An Evaluation Benchmark Assessing comprehensive performance of LLMs in highly specialized legal domains on Chinese Law
A collaboratively built large language model benchmark for legal reasoning
Provider
Stanford
Dataset name
Dataset size
162 testcases
Benchmark type
Open-ended task
Date published
August 20, 2023
Industry
Legal
The Overruling Dataset: A Benchmark for Detecting Legal Decisions that Have Been Overruled
Provider
Casetext
Dataset name
Dataset size
2,400 testcases
Benchmark type
Binary classification
Date published
April 22, 2021
Industry
Legal
