{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e8652885-31f9-5c01-95d1-46541b503393","task_key":"default--validation--31184","task_revision_id":"1","upstream_id":"31184","short_description":"HellaSwag validation 31184","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to contact a girl who's mad at you [title] Ask friends and family for their honest opinion. [step] Don't use this time to gossip or spread false narratives about the situation. Only try to find the truth of what's happening and determine what the cause of the fight is.\",\"endings\":\"[\\\"It's important to know why she isn't talking to you. This knowledge will help you know how to approach the situation in a positive way.\\\", \\\"If they say' i know ,' respond differently. They may have previously said bad things to you or done something that didn't directly offend you.\\\", 'This will help you avoid getting annoyed with the girl and get a better sense of how she will behave once she calms down. [substeps] Share some of the details of the fight with some friends.', \\\"This will convey to the girl that you're the one, not that some way outside her control is playing a role in the scene. Don't let this prejudice ruin your friendship with her, but also avoid bothering her about it.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}