{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2b0b7a10-a9ec-53a7-aabd-b8e36453ae3e","task_key":"default--validation--49278","task_revision_id":"1","upstream_id":"49278","short_description":"HellaSwag validation 49278","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to make a home smell nice naturally [title] Do something that smells great, like bake bread. [step] A home's centre is often the kitchen. In this day and age, it is often the case that the whole family gets to see each other for meals and so it is important that it smells great.\",\"endings\":\"['To do this, grab your favorite sandwich, a drink of soup, or a handful of lavender sticks from the kitchen cupboard, and do some rosemary breath. Some rosemary include anise (rosemary is a powerful, calming fragrance).', 'Avoid organizing your home by setting up a central area, such as the living room, kitchen or the bedroom. As you get older you may discover over time that it has become a weekly feature.', 'Bring a baking sheet or a pillow for each member of the family to place on the house and sit on while cooking. [title] Make your sheets a three dimensional effect of your home.', \\\"The smell of baking bread is amazingly good. If you don't have the time to bake your own, buy some from the bakers.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}