{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"10108633-f6c5-5892-a93d-8c49d76141ce","task_key":"default--validation--48854","task_revision_id":"1","upstream_id":"48854","short_description":"HellaSwag validation 48854","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to find a missing pet [title] Search your home. [step] Often times a pet will hide somewhere in the house if he becomes startled or upset. Before you panic and start canvassing the neighborhood, do a thorough inspection of your home to see if your pet is hiding somewhere nearby.\",\"endings\":\"[\\\"[substeps] There's a good chance your pet might just be hiding. Many pets do this when something frightens them or if they feel threatened in any way.\\\", 'Search around the perimeter of most of your house-outside doors, windows, in around closets, and around garage doors. [substeps] Check under doors, under foundations, and under gas or fire exits.', 'You might also check under couch cushions, under your bed, under bedside drawers and under the underside of other furniture. [substeps] If your dog disappeared through the back door, move it.', '[substeps] Examine the exterior walls and exterior trim. Windows are often closed up but a pet might be outside.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}