{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"defafa1c-6646-5bc1-a6e3-ee2d302d85c8","task_key":"default--validation--35705","task_revision_id":"1","upstream_id":"35705","short_description":"HellaSwag validation 35705","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to get your best friend's boyfriend to like you [title] Smile. [step] He'll likely be nervous about making a good impression with his significant other's best friend. Put his mind at ease by flashing a big smile when you meet him.\",\"endings\":\"[\\\"It's not unusual for a shy guy to grin like a jack o' lantern when meeting a girl. [substeps] He may get nervous if the beautiful girl he meets takes his breath away but never cracks a smile.\\\", \\\"If you try to smile too easily, he'll feel uncomfortable and forget to trust you. [substeps] This shouldn't make you fake or look phony; put it on for him by making sure your facial expressions are real.\\\", \\\"Also, whenever you catch his attention, ask him if he's ready to spend time with you in the future. [substeps] Be aware that people are more likely to be offended when you smile.\\\", 'A genuine smile is perceived as a sign that you like someone and want to be friends with them. [substeps] In a genuine smile, the corners of the mouth are pulled upward and the muscles around the eyes contract.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}