{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f3d89785-cce5-5587-95cb-1b7372fde59f","task_key":"default--validation--23710","task_revision_id":"1","upstream_id":"23710","short_description":"HellaSwag validation 23710","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to work with children [title] Tell the children the goal they are to achieve through the task. [step] Children are often very anxious and cautious. They like to know exactly what they will be doing or what will be happening.\",\"endings\":\"['[substeps] Always give them the reassurance of stating the objective. Show a picture or sample of the completed task when working on a project with a child.', 'Be straightforward with them, giving as much as you can about how they will reap the rewards. [substeps] It may seem like a typical task and they may think you are trying to get them to do it themselves.', 'Limit their anxiety and make sure they know for sure what they are going to be doing. [substeps] Make sure all the kids are given a list of things they need to do in the \\\" new task \\\".', 'Try to answer all questions honestly, when possible. [title] Help children get a sense of right and wrong.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}