SpreadsheetBench
Renmin University of China (RUC)
912 real Excel-forum questions, each tested on multiple spreadsheet files, to measure LLM spreadsheet skills.
A benchmark built from questions asked on online Excel forums, with their spreadsheets. One row is an instruction with its spreadsheet files and expected outputs. Evaluation resembles an online judge: each instruction has about three test files, so a solution must work generally.
Access
- Cost
- FreeNo charge.
- License
- Open licenseCC BY-SA 4.0
- Sign-up
- No sign-up
- How you get it
- Bulk downloadJSON, XLSX, Tar archive
What's in it
- One row is
- One row per instruction with test spreadsheets
- Covers
- Not geographic
- Years
- Forum posts (see paper)
- Updated
- One-time release
- Size
- 912 instructions and 2,729 test cases
Quality
Real questions with robust multi-file tests, and a successor version exists.
- Forum spreadsheets may contain personal or business data
- Download is a tar archive rather than a Parquet table
Connects to
Questions it could answer
- Which instruction types have the lowest success rates?
- How much does a model's result change on different test spreadsheets?