{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"98936bff-f54e-5367-aed6-311e793e1863","task_key":"default--validation--24782","task_revision_id":"1","upstream_id":"24782","short_description":"HellaSwag validation 24782","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"A team of basketball players are shown on the right side of the court doing lay-ups. after several turns\",\"endings\":\"[', they move to the left side of the court and begin practicing the lay ups once more.', 'the team takes off their helmets and tennis shoes.', 'the game goes very well and many people run onto the court and watch young boys do various sports on the court.', 'a man comes into frame and begins demonstrating the steps to pull the ball up his body.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}