{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"45fb3f39-a7d3-5e35-bc6d-3f18829b1216","task_key":"default--validation--50568","task_revision_id":"1","upstream_id":"50568","short_description":"HellaSwag validation 50568","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to rent a limousine [title] Decide when you'll rent the limo. [step] Early summer is the peak of the season for hiring limousines. You will probably be able to secure a better price if you hire a limo outside the months of may and june.\",\"endings\":\"[\\\"You're also likely to get a better rate on weekdays. [substeps] Sunday through thursday are typically the least expensive days, while fridays are about 20% less than saturdays.\\\", \\\"If you're looking for one in july or august, you might want to be the driver. [title] Decide whether to ask friends, relatives, or strangers.\\\", '[substeps] Many limousine offers large limousine packages for lower prices. However, the driver hiring a limousine for the funeral may not be the first limo you hire.', 'A limousine is usually on the streets or across city traffic and less hectic between the ages of september and july. Saturday taxis are typically used if you live in an area where it is hot.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}