{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"60912fb9-2afe-5427-9493-683e037a6bcf","task_key":"default--validation--43802","task_revision_id":"1","upstream_id":"43802","short_description":"HellaSwag validation 43802","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to prepare to design or organize a space [title] Get a good notebook to write in and keep all of your work organized. [step] 7.5 \\\" x9 \\\" graph paper moleskin or piccadilly notebook are good choices. [title] Define the space that you want to design.\",\"endings\":\"['[step] Whatever the location of the room, or your intended theme (i.e. try to sketch an octagonal room with the sony or ballcoco look to it), your theme should cover it all.', '[step] Say you want it to have a grid with racks, shelving and shelves. How does it look? Who will look at it? [title] File the area where you will place the shelves.', '[step] Design garage [title] Begin a goal or to do list of things that you would like to have completed with the space that you want to design. [step] Add to it whenever possible throughout the entire process (usually good ideas come to us at random times so always have your notepad at your side for quick brainstorming).', '[step] For example, if you want to design your room with pictures, use that space as the base. If you want to use lots of pictures, consider no more than 7.5 \\\" x11 \\\".']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}