{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1a8387a9-769e-56a2-a612-d27ac4b92497","task_key":"default--validation--7449","task_revision_id":"1","upstream_id":"7449","short_description":"HellaSwag validation 7449","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to be a brilliant student in history [title] Brainstorm. [step] Cover the things that you don't like about history, or have difficulties understanding. You might even link two or three ideas together if they are caused by the same reason.\",\"endings\":\"['That way, you can apply the concepts or ideas to specific topics. [substeps] Think about what you like about the news and play it at your core.', '[substeps] Put together class notes, draw up an idea or even instorm list names, and brainstorm ideas. Shut down hard concepts and draw up a map of the hall and field.', \\\"[substeps] Don't hesitate, scribble everything down. Even though in normal classes, you are told to write down the important facts, there is much more to history than meets the eye and those small facts may come in handy.\\\", '[substeps] Once you have discussions, break them up into groups, and come back to them during the meeting. Spend a few more minutes on goal to say what you are going to say.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}