{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7a100e05-d561-5658-b007-d2c5652ebc85","task_key":"default--validation--11877--variant-1adaba40443a","task_revision_id":"1","upstream_id":"11877--variant-1adaba40443a","short_description":"HellaSwag validation 11877","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to have a good time at sleep away camp [title] Visit the camp website. [step] It provides valuable information regarding rules, cabins, and what you do at camp. If there are any free camp brochures, those are useful too.\",\"endings\":\"['[title] Meet the counselors and camp counselors at several different locations, almost at once. [step] This is to help get an idea about who does the camp and what stuff you need to be know.', \\\"If you know anyone who goes to (or has gone to) the same camp, ask them questions. Get your parents' help in calling or emailing the camp director with any questions that you haven't been able to find the answers to.\\\", '[title] Maybe from your local national center, see if the new side camp will offer your camp camp kit. [step] Pack fuel in perishable food, water, food.', '[title] Join the camp at least once a week. [step] Once a week, check out the possible courses, and if the camp has one, when you are doing your orientation.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}