{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c77e02dd-336d-5d37-870b-92ee8fcf4d9b","task_key":"default--validation--23760","task_revision_id":"1","upstream_id":"23760","short_description":"HellaSwag validation 23760","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to raise a toad [title] Catch your captive. [step] Go out either early in the morning or at night, and look closely. Toads use camouflage, so it might be hard.\",\"endings\":\"[\\\"Most toads will hide close to their dens in a corner, but otherwise, they will very likely leave their den when you approach. If your captive does so quietly, the toad won't notice your presence.\\\", 'Toads usually burrow deeper into other areas to hide. There will be larger toads surrounding what they are hiding.', 'They also like hiding under leaves, behind pieces of wood, and around large rocks. When you find it, scoop him up in a net or bag of some kind, and take him to his new house.', 'If you see a toad looking so prey, approach it calmly and slowly. If it runs away or hides, that might be an affirmative sign.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}