benchmarks.wiki

benchmarks.wiki / Public workspace

JailbreakBench

Safety

JailbreakBench is an open robustness benchmark for jailbreaking attacks, defining 100 harmful behaviours alongside 100 matched benign ones. This catalogue deliberately introduces and links it rather than reproducing any of its prompts.

jailbreakbench

Official source

Download answersVisit the official benchmarkWhere the answers areStart a benchmark discussion

History

The JailbreakBench project introduced an open benchmark in 2024 for evaluating attacks on language-model safeguards and the defences against them. Its behaviour dataset was published on Hugging Face in June 2024. The evaluation includes benign requests as well as harmful targets so that robustness can be considered alongside unnecessary refusals.

How it was built

The source defines 100 harmful behaviours and 100 matched benign behaviours, which serve as targets for attack and defence evaluation. A judge model assesses responses against the defined behaviours. Attack success rate measures harmful compliance, while the benign behaviours support a separate assessment of over-refusal.

At a glance

Metric
attack success rate, with the benign pairs used to measure over-refusal
Size
100 harmful and 100 benign behaviours in the source; none imported.
License
mit (declared on the Hugging Face dataset card)
Released
Dataset published on Hugging Face 2024-06
Maintainer
JailbreakBench (source organisation)
Answer access
No problem text is imported, so there is no answer storage for this entry. The source distributes the answers under its own terms.

What a problem looks like

Paired harmful and benign behaviour descriptions, used as targets for attack and defence artefacts.

How it is scored

Scored by a judge model against the defined behaviours; nothing is imported here.

Metric: attack success rate, with the benign pairs used to measure over-refusal

Why it is hard

The benign twins are the point: a defence that simply refuses more often scores worse, so the benchmark measures discrimination rather than caution.

Limitations and criticism

A finite collection of behaviours cannot establish safety across all possible requests or attack strategies. Attack success rate depends on the attack method, its budget, the target model configuration and the judge used to classify responses. Judge errors can affect the reported rate, and low attack success alone does not establish a useful defence without considering performance on benign requests.

Cite

@article{chao2024jailbreakbench,
  title={JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models},
  author={Patrick Chao and Edoardo Debenedetti and Alexander Robey and Maksym Andriushchenko and Francesco Croce and Vikash Sehwag and Edgar Dobriban and Nicolas Flammarion and George J. Pappas and Florian Tramer and Hamed Hassani and Eric Wong},
  journal={arXiv preprint arXiv:2404.01318},
  year={2024}
}

Discussion

1 thread and 0 posts on this page. Anyone may reply; no account is required.

Contribute

Share a reproduction, report an evaluation issue, or suggest a correction to this benchmark overview.

Public problem summary

Catalogue metadata only; public problem statistics are not available.

Public problems
—
Formats
—
Problems with images
—
Answer availability
—
Configs
—
Splits
—
Awaiting a first discussion
—
Problems with a discussion
—
Public contributions
—