benchmarks.wiki

benchmarks.wiki / Public workspace

Berkeley Function Calling Leaderboard

Agents / Tools

The Berkeley Function Calling Leaderboard evaluates tool use. It works up from a single call to parallel calls, multi-turn dialogue, and recognising when no function should be called at all.

bfcl

Overview onlyNot imported on this site

Official source

Overview only: problem text has not been published on this site; the catalogue records the benchmark introduction, category and official source.

Visit the official benchmarkStart a benchmark discussion
Catalogue status
Not imported on this site

At a glance

Size
Several thousand evaluation cases across the categories; none imported.
License
apache-2.0 (declared on the Hugging Face dataset card)
Released
Dataset published on Hugging Face 2024-02
Maintainer
Gorilla LLM, UC Berkeley (source organisation)
Answer access
No problem text is imported, so there is no answer storage for this entry. The source distributes the answers under its own terms.

What a problem looks like

A user request with a set of callable function signatures; the expected output is the call, not prose.

How it is scored

Scored by abstract-syntax-tree comparison against the expected call, and by executing the call where the category allows it.

Why it is hard

Getting the function right is not enough — the arguments, their types and the decision not to call anything all count, and the last of those is where most models lose points.

Discussion

Contribute

Share a reproduction, report an evaluation issue, or suggest a correction to this benchmark overview.

Public problem summary

Catalogue metadata only; public problem statistics are not available.

Public problems
—
Formats
—
Problems with images
—
Answer availability
—
Configs
—
Splits
—
Awaiting a first discussion
—
Problems with a discussion
—
Public contributions
—