{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"016dd534-55de-5b8d-a05f-7e4f9dfb8aca","task_key":"default--validation--5352","task_revision_id":"1","upstream_id":"5352","short_description":"HellaSwag validation 5352","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to identify and treat recluse (fiddleback) spider bites [title] Find the spider that bit you, if possible. [step] If you can, try to trap the spider that bite you to determine if it is a recluse spider. You can also try to remember the appearance of the spider that bite you.\",\"endings\":\"[\\\"A recluse spider is tan all over, with legs attached to the upper part of its body. [substeps] You don't have to catch the spider.\\\", 'It can be a gradual crab bite, a cold bite, or a combination of these. If it bit you, then you can keep the spider in a closet or corner and try to catch it.', \\\"[substeps] If you can recognize the spider's appearance, try to recall what it looked like when you were young and how it looked on you. A recluse spider is a young spider, usually about 5 to 7 years old.\\\", '[title] Keep an eye out for the spider if you ever see it at the end of your driveway. [step] If you ever see a spider in your driveway, it will probably return to a place where it can hide.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}