{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fc4b59c8-0a63-5ee8-a018-a3c1d61b4759","task_key":"default--validation--19665","task_revision_id":"1","upstream_id":"19665","short_description":"HellaSwag validation 19665","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to check your psat score [title] Wait six to eight weeks after you take the exam. [step] The psat scores are released a couple of months after you take the psat. The psat is given in october, so the results are usually available the beginning of december.\",\"endings\":\"['Schools generally get the scores a week before they are released online. [title] Find the access code on your exam score report.', 'You should get your psat scores before the deadline. The psat score will be impacted based on whether you take the test multiple times, or have been disciplined for failing the test.', \\\"[substeps] In order to keep your psat score up, you'll need to be in touch with your sponsor, or contact your coach in advance or have a positive reply submitted. Testing generally costs anywhere from 1 to 3 weeks; after this time, you'll proceed to another stage.\\\", 'If you take the psat at the last minute, you may have trouble remembering the date. A fresh psat score is likely to be crucial, so prepare ahead of time.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}