{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2a93f7be-9c44-5b66-b782-1ca08283570e","task_key":"default--validation--31547","task_revision_id":"1","upstream_id":"31547","short_description":"HellaSwag validation 31547","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to respond when your kid bosses you around [title] Remind them of their limits. [step] Tell your child that disrespect out out-of-bounds, especially with you. Tell them what kind words they can use and what words are not okay.\",\"endings\":\"['Try to be heard and take their point seriously. They will learn to respect you better in the long run.', 'Treating you with disrespect is not okay. [substeps] Say, \\\" it\\\\'s not okay to talk to me that way, it\\\\'s disrespectful.', 'Try to explain and correct the underlying reason for violating a small rule. [substeps] For example, you could say, \\\" sam, you have an obligation to keep feeding your dog, and yours has to do so every day for ten years.', 'Focus your voice on one area and then say \\\" last time i said that, you said...\\\" or \\\" you yelling, pushing, working on this. \\\" communicate your feedback to your child by emphasizing one area while maintaining your very strong demeanor.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}