{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c2981927-3e9b-5c11-a38e-874061e4173e","task_key":"default--validation--36283","task_revision_id":"1","upstream_id":"36283","short_description":"HellaSwag validation 36283","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to peel potatoes with an ordinary kitchen knife [title] Place your potatoes on a cutting board near the side of your sink. [step] Place a colander on the other side of the sink. If you don't have a colander, then place folded paper towels or a kitchen towel opposite the cutting board.\",\"endings\":\"['[substeps] Unfold the paper towels to reveal a pad of paper towel. Place the folded paper towel on the pad of paper towel and trim away any excess paper as you slide the chips off.', 'These will absorb the water that collects after you wash the potatoes. [title] Rinse each potato under cold running water as you scrub the potato with a sponge or a nylon bristled brush to loosen dirt and debris.', \\\"Place your potatoes on this towel so that they don't twist. [substeps] Note that you cannot put butter into the colander.\\\", \\\"[title] Place your peel on the paper towels. [step] Just use any sort of paper towel that isn't too wet.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}