{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"52343bfb-73be-5d37-b397-cf4a97f1a8db","task_key":"default--validation--47836","task_revision_id":"1","upstream_id":"47836","short_description":"HellaSwag validation 47836","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to diagnose a learning disability [title] Evaluate the person's current behavior. [step] If you think you or your child may have a learning disorder, consider what they are doing to make you think that. Take note of what you see and hear from your child that you believe may be a symptom of a learning disability.\",\"endings\":\"['[substeps] If your child generally communicates or talks very differently during the day, he or she may be deaf or hard of hearing. If your child is unable to communicate at all during the day, this may be a symptom of a learning disability.', '[substeps] Such assumptions often include judging or criticizing the person when they do things incorrectly, e.g., if they do not follow a timetable for the day or even does not pay attention to how they do things. [title] Ask the person you suspect of learning disabilities about their behavior.', 'You may also be able to identify signs of a learning disability and are concerned to help the person when it arises. [substeps] You may notice that you or your child are often ignored or interrupted in class.', '[substeps] Write down any behaviors or challenges your child faces that you believe may be unusual. This will be useful when you talk to a professional.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}