{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"87df7bc5-683c-5ef7-84b5-72a731830299","task_key":"default--validation--15564","task_revision_id":"1","upstream_id":"15564","short_description":"HellaSwag validation 15564","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to answer a request for admissions [title] Find your deadline for responding. [step] You can't miss the deadline for responding to the requests, so you should quickly find how much time you have to respond. Look at your rules of civil procedure and find the rule on requests for admission.\",\"endings\":\"[\\\"It doesn't necessarily mean you need to reply right away. Rather, it means you have to respond within six months.\\\", \\\"[substeps] If you are in federal court, then federal rule 36 is the relevant rule. If you are in state court, then the rule will depend on your state's rules of civil procedure, which you can find online.\\\", '[substeps] Usually there are multiple deadlines for responding to requests. The time limit often differs by state and country.', '[substeps] The rule of civil procedure is usually detailed online, or you might find it at the lower left of your page. Generally, you must admit or deny each allegation sent to you by opposing counsel.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}