{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c25ac5a5-9354-5483-9653-f6e7c4d92c55","task_key":"default--validation--33216","task_revision_id":"1","upstream_id":"33216","short_description":"HellaSwag validation 33216","config":"default","split":"validation","body":"{\"activity_label\":\"Blow-drying hair\",\"ctx\":\"The girl blow dry her hair with white blower, sectioned her hair, brushed her hair with roller brush while blow drying it at the same time, she roll the brush downwards and upwards. she\",\"endings\":\"['styled her hair by brushing and combing it to give more volume.', 'put complete hair style to her hair, then she turned around to show the finished hair look, she smiled.', 'blow dry her hair to make it shiny.', 'also blow dry her hair to effect features of curls along her head, and also blow dry her hair from front to back.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}