{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7f58c725-d4d3-51d3-97d4-2b0e21e4e276","task_key":"default--validation--6115","task_revision_id":"1","upstream_id":"6115","short_description":"HellaSwag validation 6115","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to get everyone to do one's chores [title] Discuss and divide the chores. [step] Every household has certain chores that need to be done. The first step to making sure they are all done on a regular basis is to identify them all.\",\"endings\":\"['Issue a preference for the chores that should all be done by each child. [substeps] You might say, \\\" we all need one chore each day.', 'Sit down with all the members of your family who are old enough to do chores and work it out. Letting everyone have a say in their chores and getting them all to agree on their chores will help you get everyone to do them.', 'From here, you can build a hierarchy for each of the chores that has to be delegated throughout the family. Let the children act as the leader, which they feel comfortable with.', \\\"Disagreements about the chores should be evenly divided among the household. [substeps] Don't simply assign every chore to one child, invite them each time they do something they are not supposed to.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}