{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b5baf7bc-e301-5bd9-b1a2-e7cc8bf3613c","task_key":"default--validation--36806","task_revision_id":"1","upstream_id":"36806","short_description":"HellaSwag validation 36806","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to choose a school [title] Get the test scores. [step] Test scores are not the whole picture, but many parents will start out there. [substeps] Determine the online source for school rankings.\",\"endings\":\"[\\\"Get a copy of your sat scores for each. Run your old sat scores through the student's defense scores.\\\", \\\"These will be the best news sources for the teacher. Some good sources to look for in a math or humanities course include university's website, textbooks, online copy of websites like one, and psychology websites like other university students'.\\\", 'It is different in different states and countries, but you will find it if you search online for \\\" school rankings for your location. \\\" for example, in california, usa, every school has its ranking by test score listed on california department of education website.', 'Links to schools through online sources will usually sit down to look over the list of stellar marks awarded and given by superschools. Compare the scores to the classroom ratings ratings or class load report.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}