{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f46f4c19-1550-5e8b-900a-570531533b0b","task_key":"default--validation--35739","task_revision_id":"1","upstream_id":"35739","short_description":"HellaSwag validation 35739","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to plan an arab wedding [title] Pick the hall. [step] The reception hall is crucial and must be reserved at least six months in advance. When looking for the perfect hall, size is the most important factor.\",\"endings\":\"['A research group will have recommendations for the ideal hall that the bride and groom can agree upon. [substeps] Indication columns will discuss the length of the hall.', 'Every table needs to be a fair height and equally spaced throughout. [title] Decide if you want to go big or small.', \\\"It should be spacious enough to seat the engaged and distant muslim bride and groom. Additionally, there should be enough music for the reception and for all the guests you're inviting.\\\", 'A hall should be large enough to fit anywhere between 500 to 1 , 500 people. Décor is simply extra and not viewed as a main priority.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}