Benchmark AI

Benchmark AI / Public workspace

BIG-Bench Extra Hard (BBEH)

Reasoning

BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.

Japanese introduction

カテゴリ別の難しい課題を通じて、言語モデルの推論能力を評価するベンチマークです。fullとminiの配布があり、この取り込みではfullの論理式カテゴリ200問を扱います。

bbeh

Full text200 tasksProblems imported

Official source

Full text: eligible public problems include text, images and discussion.

Tasks

Tasks

Whole public benchmark.

Public problems
200
Formats
text (200)
Problems with images
0
Answer availability
Answer published by the source (200)
Configs
bbeh_boolean_expressions (200)
Splits
full (200)
Awaiting a first discussion
200

Discussion