{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"881592bb-1222-5e4b-8eff-ab48e39847df","task_key":"default--validation--5769","task_revision_id":"1","upstream_id":"5769","short_description":"HellaSwag validation 5769","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wrap african head wraps [title] Open wide the head wrap or headgear from length to length. [substeps] If it is too wide, fold it into another triangle. This size will be okay to fit to your head.\",\"endings\":\"['Now you will have the longer side and the tailed and pointed side. [title] Hold the longer side from one palm to the other.', '[title] Press down with your thumb on the side that faces you. [step] This creates a long necklace and wrapping times will generally be 2 to 3 full minutes.', 'Alternatively, get a heavy staple gun and staple to the bottom of the head wrap first for extra security. [title] Cut holes for the ear pieces and the eyes.', 'Ensure that the eye hole is just through the opening of the body wrap. If it is not very wide, gently pinch the ends from one end and pull them through each from the other end to form an open triangle.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}