{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3430f52f-8b9e-5bba-bb21-0bbf570a0b83","task_key":"default--validation--7989","task_revision_id":"1","upstream_id":"7989","short_description":"HellaSwag validation 7989","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to succeed in clash of clans [title] Be careful with gems. [step] This is the most important and rare resource, so use them wisely. Use your gems to speed up barracks production, or use them on collectors.\",\"endings\":\"['[title] Use more than two gems when playing team fortress 2. [step] Wear the one with the most mana and the strongest quest room (tower) to flatten players.', 'Market it to warriors who want your heirloom. Do not sacrifice it to children to combat your customers, or to thieves, or to dwarves! [title] If you have campaign spots, once you get attention, level down to harvest.', '[title] Train until your army is an oval in size (gildehyde by heart speech, which builds a variety of colony rooms). [title] Train tightly to the point of victory.', \\\"But be careful when deciding to use them, and make sure it'll be worth it. Don't spend them on hurrying troops' training unless it's absolutely necessary.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}