{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c6f2fafb-f2a2-59f1-a3b2-63d6c6e27681","task_key":"default--validation--12393","task_revision_id":"1","upstream_id":"12393","short_description":"HellaSwag validation 12393","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to call \\\" seat check \\\" [title] Decide to leave your seat and recognize the need to seat check. [step] Many people do not realize this, but the purpose of the seat check is to prove that you realize that your seat must be protected and establish \\\" dibs \\\" (and therefore the moral high ground). Therefore, you must call seat check before leaving your seat.\",\"endings\":\"['This is sometimes called standing check or \\\" host call \\\", and sometimes called \\\" footers \\\" or \\\" diners. \\\" you do not have to leave your seat to perform this activity.', 'Attempting to call it from across the room or after you have stood up, as someone rushes into it is extremely lame and forfeits the entire point of the check. [title] Make sure that at least two people hear you call seat check.', '[title] Visit https: // www.seatseat.com/seat-check in a web browser. [step] To make an unrestricted seat check, visit.', '[title] Locate the pull out lever next to the seat. [title] Let the lever go back, meaning it will not let go, when you exit your seat.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}