{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"87e71d79-1bfa-5c7f-9a19-998db248684c","task_key":"default--validation--3395","task_revision_id":"1","upstream_id":"3395","short_description":"HellaSwag validation 3395","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to cross examine [title] Have command of the case. [step] To an outsider, a cross examination might seem like a series of random questions, but the process is actually incredibly well-planned and requires hours of preparatory work. It's essential to know the ins and outs of the case in order to ask just the right questions.\",\"endings\":\"['Start conducting research for the cross examination as far in advance of the trial as possible. [substeps] Learn all the facts of the case, not just those you need to know before the trial begins.', \\\"Remember to stay mentally unemotional and to remain direct without emotion. [substeps] It's important to take responsibility for the academic or historical research that you're working on when you're doing the cross examination.\\\", '[substeps] You can test out a case, or search online to find examples of cases to cross examine. Open your case and narrow the search to the basics, such as whether the case has little to no consumer protection or compliance with a government requirement.', \\\"[substeps] Identify the person who has the most points. While bringing an opinion can be helpful, it's actually designed to be skeptical, especially in a subject as big as the trial for which you will be asking the judge.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}