{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d56790a2-b60c-5302-be07-4e302d4c4d71","task_key":"default--validation--8871","task_revision_id":"1","upstream_id":"8871","short_description":"HellaSwag validation 8871","config":"default","split":"validation","body":"{\"activity_label\":\"Baton twirling\",\"ctx\":\"In a gym a baton twirling troupe walk into a gym and get into their places as they prepare to do a routine. the music\",\"endings\":\"['resumes with the multiple open stances, flips, and shoulders.', 'begins and they begin to do the routine on a pink and yellow springboard.', 'plays behind them as they move their hands around and around on the exercise equipment.', 'begins and the girls do their routine of dancing, jumping, making different line formations and baton twirling as the crowd cheers them on.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}