{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ab8665c8-4b9d-5c60-a800-55e4e61636b4","task_key":"default--validation--6180","task_revision_id":"1","upstream_id":"6180","short_description":"HellaSwag validation 6180","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"Two young girls appear in the bathroom, one has a green headband around her forehead and the other has paint across her face. both of the girls\",\"endings\":\"['then start fighting with each other, screaming and throwing food in  front of the mirror.', 'are laughing and joking with one another until they start to wash their face and clean it with green rags.', 'begin to play on the water using tape and squeege drums to play underwater in different poses.', 'walk in and out of frame polishing different surfaces of the bathroom, brushing their nails, and making cracks.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}