{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"47fdbccc-9b7b-5276-a2d4-bf16357c5cbf","task_key":"default--validation--15001","task_revision_id":"1","upstream_id":"15001","short_description":"HellaSwag validation 15001","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to care for several teddy bears [title] Choose their personality. [step] They could all have the same personality or they could have different ones. Choose their gender too.\",\"endings\":\"['[title] Gently pick out teddy bears that are similar in shape and appearance. [step] You can even paint their face and limbs with stencil designs.', 'This may seem like a lot of work, but it is the best way to assess their personalities. You cannot always find the gender of your teddy, but you can always seek out and use their looks in your searches.', \\\"Always check the medium (irc) scale in packages. Choose one teddy, and don't choose two.\\\", '[title] Make or buy them all clothes, if you want. [step] Buy them all the same amount of clothes, or else one might feel like you love the others more! [title] Give each of them at least one toy of their own.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}