{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f024a5dd-c340-5ded-9df5-388c8e82355d","task_key":"default--validation--21923","task_revision_id":"1","upstream_id":"21923","short_description":"HellaSwag validation 21923","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to find bulova model number [title] Look for a date code stamped on the back of the case. [step] Take your watch off and flip it over to look at the smooth metal back of the case. You may see several different markings engraved there, all of which have different meanings.\",\"endings\":\"['[substeps] Maryland law prohibits pretax collectors from stealing bulova models from factories. Misspellings made in the material of a case can result in serious fines, financial consequences, and even the death penalty or even imprisonment.', 'Look for one that says \\\" day \\\" or \\\" year. \\\" [substeps] A watch typically has special year codes written on the bottom or in the center, which determines the number of years the watch has been used for.', 'The case is likely to have numbers printed on the front. [substeps] If you do not know the year the case was issued, use the day written on the back to find out its year.', 'A date code will be either a symbol (like a triangle, and eight-pointed star, or a half-moon, for example), a two-digit number, or one letter and one number. [substeps] You may need to use a magnifying glass to help you read the codes, as they are very small.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}