{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"144d1df1-2446-550d-bc41-c30db4324621","task_key":"default--validation--17865","task_revision_id":"1","upstream_id":"17865","short_description":"HellaSwag validation 17865","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to shave your genitals (male ) [title] Trim the hair. [step] Buy a safety razor and trim the pubic area to less than 1/4 \\\". This will make the hair easier to shave later.\",\"endings\":\"['[substeps] All bushes come with opaque plastic nippers to remove pubic hair. The edges of the blade themselves are sterilized.', 'If you shave at the groin region, trim it and apply some vaseline for an even shave! [title] Look into a green shaving cream (also known as propionate). [step] These are easy to apply and a.', '[title] Place strips of shaving wax or other lubricating wax on your skin, firmly running them between your hands. [step] Do not rub them, leave them in place.', 'Longer hair is more likely to clog a razor. Using a safety razor will work better than scissors and it will cut the hair shorter.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}