{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25cb8ba1-0139-517d-9c88-84ebc2595bb2","task_key":"default--validation--34815","task_revision_id":"1","upstream_id":"34815","short_description":"HellaSwag validation 34815","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to become a dynamic person [title] Read for at least half an hour daily. [step] Take 30 minutes daily to learn more about what's going on in the world around you. Read articles to stay updated on current events or pick up that fiction novel you've been wanting to read.\",\"endings\":\"[\\\"Watch some funny videos to listen to while you're studying. Either use a free computer computer or you can purchase one at a library.\\\", 'Get your head down and spend at least half an hour on each topic. [title] Act calm, cool, and collected.', \\\"[substeps] If you've always wanted to try out a new sport, think about lifting up one of those weights at the gym. You can always measure your progress and build a workout regimen.\\\", 'Reading helps to expand your vocabulary and knowledge. [substeps] Subscribe to newspapers on your phone.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}