{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7bd17e0e-c0d4-5816-aa8e-5e578e1e07ad","task_key":"default--validation--33100","task_revision_id":"1","upstream_id":"33100","short_description":"HellaSwag validation 33100","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to make a chore chart with incentives [title] Make a list of household chores. [step] Chores in chore charts are generally things that need to be accomplished in the house daily or weekly. Think about your home's needs and which chores you prioritize more than others.\",\"endings\":\"[\\\"[substeps] For example, you may suffer from allergies and need the house thoroughly dusted and vacuumed every week, but folding laundry isn't a huge priority for you. Make sure your chore chart reflects your household's necessities.\\\", '[substeps] For example, you may set up your kitchen and have all the dishes ready. Your chores could include: dusting, washing, putting away dishes, or doing laundry.', '[substeps] Leave out the housework that needs to be done. Make small, individual entries for things like take out the trash, mow the lawn, vacuum, fold laundry, or et cetera.', 'For example, you may have oatmeal sitting at the kitchen table, or a pot of potato chips still sitting in the fridge. Prioritize these chores over other things like cooking and laundry.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}