{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c6a600c0-4c3c-574a-b7a6-f99dbd0e40ee","task_key":"default--validation--28277","task_revision_id":"1","upstream_id":"28277","short_description":"HellaSwag validation 28277","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to march [title] Stand at attention. [step] There are two possible commands for standing at attention: \\\" fall in \\\" is used to either assemble individuals into formation or return the marchers to their original lineup. \\\" attention \\\" is a command given during rest position.\",\"endings\":\"['When someone moves to the side or down the line, the person standing at attention is notified. So if the marchers move back and forth, the onlookers could easily interrupt their direction of travel.', 'You should respond to both commands by standing at attention. [substeps] Bring your heels together evenly, with the toes slightly pointed outward to form a 45-degree angle between your feet.', '\\\" shoulders up \\\" is an affirmative and followed by a disrespectful turning of the shoulders or lowering the tailbone and lowering the jaw. \\\" head up \\\" is a command given during vigorous activity.', 'You must stand at attention 24/7 whenever you plan to march using either of these commands. Beyond the formation position, you will take on this command in the order of the drill and the beat of the music.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}