{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8b9bb4df-4ce7-5ac2-bf49-8622165acbc2","task_key":"default--validation--10224--variant-05d2fe211ba7","task_revision_id":"1","upstream_id":"10224--variant-05d2fe211ba7","short_description":"HellaSwag validation 10224","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to purchase your own phone as a teenager [title] Work around the house. [step] Offer to do some serious cleaning that could take the burden off your parents...for a fee. Charge a small sum of money for each chore, depending on the type of chore.\",\"endings\":\"['Some good examples are : [substeps] Unplugging your phone. Your phone may not function at night, so turn off the television at night.', 'The more undesirable, the more you should ask for. Set a maximum asking price depending on how well set off your family is.', 'Make sure the family has specific chores in mind : [substeps] You might want to do the laundry every week after school, or email weekly to make sure that your soccer coach agrees to make necessary household cleaning adjustments. Even toddlers need to be helped by their parents first thing in the morning, before you get to bed.', 'Be sure to take advantage of any yard tasks, like weeding, raking, and dusting. [substeps] As the teenager, you might try investing in a few vacation days.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}