{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c8cbfa43-7cfa-5ebd-a6f0-d0272478af3a","task_key":"default--validation--14699","task_revision_id":"1","upstream_id":"14699","short_description":"HellaSwag validation 14699","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do an indian head massage [title] Make preparations. [step] Find a quiet place away from any distractions. Make sure the room is a comfortable temperature.\",\"endings\":\"['Light candles or hot incense make it enjoyable, if you cannot do it outside, in the heat of the room. [title] Begin the massage by massaging the back of the head with your thumbs.', '[substeps] Put on some soft music light some candles to prepare the room. [title] Have the recipient take a seat and get comfortable.', \\\"Don't get cold and place a long towel in an unused bathroom. Instead, set the temperature to 200 to 170 degrees fahrenheit (75 to 85 degrees celsius), according to protocol.\\\", 'Turn off any windows, put out some fans or put on some classical music. You should also make sure the room is free of light sources.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}