{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"938c1f8c-38ec-5771-b3b8-f646e67b751d","task_key":"default--validation--33231","task_revision_id":"1","upstream_id":"33231","short_description":"HellaSwag validation 33231","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to find a job as a preteen [title] Do extra chores. [step] This does depend on how cooperative your parents are, but if you are polite in asking and explain why you want the spending money, they're likely to listen. It's an easy way to make 10 or 20 bucks (or more, if your parents are generous).\",\"endings\":\"['You could give the whole bathroom a good scrub down, or vacuum and mop the entire house. [substeps] You could ask your mom which chores she hates doing the most, and pick those up for her.', \\\"Anything under 5 dollars (or less) earns your allowance. Here's how : [substeps] Watch a movie together at the end of the night.\\\", '[substeps] Keep some activities in mind when doing chores (e.g. going to the bathroom, doing homework at home, walking to and from school, etc.', '[title] Invest in little toys for your little children. [step] As young as possible, buy baskets of tiny toy animals with fun stuff inside them or even a cat or dog with a loose ball inside so that you can look like a kid.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}