{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8e442ac0-2c29-5c96-856c-243fd6c475dc","task_key":"default--validation--40323","task_revision_id":"1","upstream_id":"40323","short_description":"HellaSwag validation 40323","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to have a ninja personality [title] Be a problem solver. [step] The ninja had to be alert at all times and be prepared to solve complicated problems on the spot. They did not believe anything was impossible for them to figure out and they looked at obstacles as opportunities to learn.\",\"endings\":\"['[substeps] Be a problem solver by sitting at a desk and writing up problems on a notepad. Write your problems down on a sheet of paper and hand write them down one day.', '[substeps] Try to approach your daily problems with that same mindset. Along with solving problems, also practice being able to identify exactly when problems occur and what caused them.', 'You have to be a problem solver. [substeps] You have to be able to solve complex problems and always have a solution in mind.', \\\"If you want to be a ninja, you need to be good at it, and you can't just be a problem solver and try something completely out of the ordinary. Try to practice this skill daily, even in the middle of the day before you're going to bed.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}