{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5fdc91cf-ce49-5d64-aa77-e8c5aef630a9","task_key":"default--validation--36101","task_revision_id":"1","upstream_id":"36101","short_description":"HellaSwag validation 36101","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to build your own computer case [title] Decide more or less what you want your computer to look like. [step] Start off by doing some rough sketches followed by more intricate designs. Think about the material you will need to use to achieve your design.\",\"endings\":\"['Your computer case should make some very large, intricate designs. [title] Repurpose a plastic cd mold of your design to reuse as a case.', 'Will it be cardboard, wood, metal, acrylic, glass? Plan out what colors you will use, even down to small details such as led lights, and graphics. This will allow you to revise your design and get it to a point you will be happy with.', 'A quilted laptop case is much more expensive than one made from a durable fabric and hard plastic. Models are usually sold in bundles of one or two hundred dollars, though some expensive computers require more or less money.', '[title] Choose the style you want your case to be made from. [step] You can use a pencil and a ruler for accuracy.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}