{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f9730a0c-64dd-560b-a1a5-98ebfbc161a5","task_key":"default--validation--4432","task_revision_id":"1","upstream_id":"4432","short_description":"HellaSwag validation 4432","config":"default","split":"validation","body":"{\"activity_label\":\"Horseback riding\",\"ctx\":\"In between interviews they show the duggar mom and the little girl getting instructions about their horse and then getting on their horses and riding on them in an enclosed area. the red and white tlc logo\",\"endings\":\"['appears on screen and all the cheerleaders begin to perform and the video overlays the footage of the cheerleaders dancing on horses all over the arena.', 'remains on the bottom right area of the screen the whole time.', 'appears on screen with pictures of the horses inside and out.', 'appears and the couple horses are shown individually, in different amounts, in different locations, and then slides down to the bottom on the table where a sign informs viewers how they should get on the horse to prepare for a ride in the saddle and the camera shakes in disbelief and disbelief.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}