{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"20ce56ee-b3ad-5348-b9e4-3a4e08649cff","task_key":"default--validation--19152","task_revision_id":"1","upstream_id":"19152","short_description":"HellaSwag validation 19152","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to keep a goal journal [title] Choose where you'll write. [step] Pick a notebook, a computer file, or whatever else you want to use for your goal journal. Make sure it's something you feel comfortable writing in and that you can have available to you when you need it.\",\"endings\":\"['[substeps] There are a number of companies that make journals specifically designed to be used as goal journals. Use one of these if you find the formatting works for you.', '[substeps] The easiest place to write in is in your desk when you have a hard time sitting down or reading, and having access to that location will always make it easier to work and yourself. You may want to carry a notebook with you wherever you go.', \\\"It doesn't have to be anything big--just something you can write and get attached to. [substeps] You can find a notebook and a computer file at.\\\", \\\"Make sure it's functional and suited to your goal. Make sure it can cope with language, but doesn't feel too cluttered or overwhelming.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}