{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"06a8550f-fafc-5ca9-9c1d-4e0d8bc4ed4c","task_key":"default--validation--23084","task_revision_id":"1","upstream_id":"23084","short_description":"HellaSwag validation 23084","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to stop your dog from begging [title] Understand the behavior. [step] Dogs are relatively simple creatures. If they perform a behavior that results in a reward, they will repeat the behavior expecting a reward in the future.\",\"endings\":\"['They will not frequently show their teeth or act immaturely. This behavior is not a sign of submission.', 'The taste of sugar, however, is more realistic than the taste of toy food or treats. Treats merely provide even the gift of \\\" happy words \\\" or little whoopies.', 'If a behavior does not result in a payoff, they will have no reason to repeat it. [substeps] Some dogs will just sit and look at you, while others might whine at you until they get what they want.', 'Treats might be an example : [substeps] A dog begs when you give it a toy. It may wait for treats at dinner to stop begging.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}