{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2c11acf8-d198-5704-a86b-c32a92214be5","task_key":"default--validation--20944","task_revision_id":"1","upstream_id":"20944","short_description":"HellaSwag validation 20944","config":"default","split":"validation","body":"{\"activity_label\":\"Tumbling\",\"ctx\":\"Two girls are seen speaking and waving to the camera that lead into them performing several jumps and tricks on a mat. the girls\",\"endings\":\"['have a drink and continue flipping around and ends with one girl standing in the end and another walking away.', \\\"continue performing and give one another high fives with one stopping to pick up other's hands.\\\", 'continue taking turns flipping all around the mat in the area as well as speaking to the camera and waving.', 'flip out and jump rope using various hands gestures and moving their bodies around each other.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}