{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0398e5bf-1edc-583b-a307-4b2a5be54355","task_key":"default--validation--25066","task_revision_id":"1","upstream_id":"25066","short_description":"HellaSwag validation 25066","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to measure chair covers [title] Identify what chairs you want to wrap. [step] You need to know whether you are measuring chair covers for only one corner piece or an entire room. [title] Determine what type of furniture covers you want.\",\"endings\":\"[\\\"[step] If you've chosen a sofa, for example, you should probably use a sofa cover to cover each side of the sofa or your entire sofa, with the sofa taking up less space. [title] Determine whether you want to wrap a duvet or comforter.\\\", '[step] Know what style and pattern you want to emulate so you know where to purchase the covers. You may also want to set an expected budget since some materials are more expensive than others.', '[step] There are two kinds of materials you could use for your chair cover : [substeps] Cabinet covers think of it as a buffet cover, but most commonly built on tables in the kitchen or dining room. This covers cover covers a regular portion of most of the table, and covers chairs directly underneath the table.', '[step] There are two primary types : [substeps] Traditional. This type of cover covers a floor, not a whole wall.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}