{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5784f542-4374-512e-a9ef-850f6cad2aa6","task_key":"default--validation--23093","task_revision_id":"1","upstream_id":"23093","short_description":"HellaSwag validation 23093","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to tell a friend that his or her partner is cheating [title] Be sure. [step] Before you jump the gun and start dialing your friend to tattle, make sure what you saw was actually cheating. You'll stir the pot for no good reason if it turns out your friend's partner is simply having lunch with a coworker or a distant relative.\",\"endings\":\"[\\\"Carefully evaluate what you saw just in case you are reading into something completely innocent. [substeps] Was there hand-holding, hugging, or kissing involved? What about the interaction makes you think your friend's partner is cheating? [title] Get firm evidence.\\\", \\\"[substeps] After you've heard the true item is a conjunction, you need to know what the other person says and how you might solve the problem. Talk to your friend or family member by asking them before mentioning about the secret affair.\\\", \\\"If your friend has a different character, you'll have to think more about whether it really is the case. [substeps] Know that in some cases how significant an assignment results in your friend finding a new partner.\\\", \\\"[substeps] Make sure that you know if your friend is going to share the secret with another person. You may have news that you didn't actually know.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}