{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6162e3dd-6943-5105-a8d1-59823c100217","task_key":"default--validation--39203","task_revision_id":"1","upstream_id":"39203","short_description":"HellaSwag validation 39203","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to help your blind or visually impaired sibling [title] Ask if they'd like your help. [step] It is best to not act like your sibling requires your assistance in every situation. Instead, gently ask if they would like your help at certain times.\",\"endings\":\"[\\\"An example would be sitting down to dinner with your sibling every now and then. This helps your sibling see that you wouldn't mind helping them treat themselves.\\\", 'For example, when your younger sibling is at school or working, you might run by and say, \\\" hi mom. It\\\\'s sam\\\\'s birthday this afternoon.', \\\"If they are in an ambulance, you can probably get them outside or to the hospital. [substeps] This type of request can still get your sibling in harm's way.\\\", 'It is best to do this in the moment or, perhaps, in advance if it is a unique situation. [substeps] For example, you might say, \\\" i notice that we are about to walk down a series of stairs.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}