{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7d608a36-0c0c-51c9-a8b8-f3f47607bb69","task_key":"default--validation--33215","task_revision_id":"1","upstream_id":"33215","short_description":"HellaSwag validation 33215","config":"default","split":"validation","body":"{\"activity_label\":\"Blow-drying hair\",\"ctx\":\"The lady sat at the center of the studio with her wet hair, she put a white cream on her hand then rub it on her hair. the girl\",\"endings\":\"['brushed her four teeth with the dark cream and finished with a pink lip.', 'then blow dried her hair and straighten the hair with a comb where the travel record indicated.', 'then hold on to the back of the curls laying the wet hair on her plates and put the piled hair under the brush.', 'blow dry her hair with white blower, sectioned her hair, brushed her hair with roller brush while blow drying it at the same time, she roll the brush downwards and upwards.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}