{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ff5b3cf8-cbe7-55aa-a15d-abd0fd741ec8","task_key":"default--validation--29512","task_revision_id":"1","upstream_id":"29512","short_description":"HellaSwag validation 29512","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make strawberry water [title] Wash and hull your strawberries. [step] \\\" hulling \\\" basically means removing the core of the strawberry and the green leaves at the top. This is done instead of simply chopping the top of, because it allows you to remove the leaves at the top (which you surely don't want floating in your strawberry water) without wasting any of the delicious berry.\",\"endings\":\"['And in this case, wash and hull your strawberries, since doing so increases their juice levels. [substeps] Using a knife or a nylon manicure brush, trim one side of the strawberry.', '[substeps] Rinse and hull your strawberries. After rinsing, run the berries under a cool faucet.', 'Make sure you wash or hull your strawberries when they are dark and dry, since these fruits, although almost fully ripe, are still underripe. [substeps] Peel the tops and cut them when they are sliced dry.', 'The easiest way to do this is to use a regular drinking straw. [substeps] Position your straw at the bottom of the strawberry.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}