{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fb5ae1f8-5cc6-524d-9830-712669332030","task_key":"default--validation--42850","task_revision_id":"1","upstream_id":"42850","short_description":"HellaSwag validation 42850","config":"default","split":"validation","body":"{\"activity_label\":\"Shot put\",\"ctx\":\"The man grabs the ball and takes a big step behind him and throws the ball. The man twists his body a few times as he holds the ball against his neck. the man\",\"endings\":\"['does a 90 degree move around as he holds the ball against his neck.', 'then extends both arms up and bows.', 'then manages to candy cube the ball and throws it on the other side of the court.', 'throws the ball to its opponent who easily catches it and throws it back to the matches.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}