{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"edfea7b0-2b51-5a1f-92b9-9621db4b000d","task_key":"default--validation--45063","task_revision_id":"1","upstream_id":"45063","short_description":"HellaSwag validation 45063","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to tell if you are an introvert or an extrovert [title] Consider your reactions around people. [step] How do you act at a party, or around a large group of people? [substeps] If you tend to be more of a listener than a talker, you might be an introvert. You would be more of a wallflower than the life of the party.\",\"endings\":\"['There is a fine line between being a wallflower and being an extrovert. Introverts tend to get emotional and get angry quickly.', '[title] Understand you may feel anxious, overwhelmed, or risky. [step] Extroverts tend to think in big numbers about what to say and what not to say.', 'A shy or introverted person probably tends to think about immediate matters, such as preparing food and activities, and then letting things roll by. A nervous or introverted person probably might not like that too much.', 'If you are more talkative, you are likely to be an extrovert. People are drawn to you because you smile more and are comfortable to be around.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}