{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"12e507cf-c29b-5a3b-a83e-7e8822728c67","task_key":"default--validation--19703","task_revision_id":"1","upstream_id":"19703","short_description":"HellaSwag validation 19703","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to be a pacifist [title] Being a pacifist is not about the label. [step] If you want to be a pacifist, it's good to start with identifying your reasons why. Try not to jump into it for status or being able to tell all your friends that you've become a non-violent person.\",\"endings\":\"['There are a few good reasons to being a pacifist. Some things you might consider are : [substeps] Becoming a pacifist establishing your own identity self-respect self awareness emotional desires a sense of pride [title] Think about your type of lifestyle.', '[title] Distinguish between hardcore and emo punk. [step] Bands that are displayed in commercially-produced clothing stores are hardcore punk.', '[substeps] Check to see if your parents approve of your use of pacifiers. Have you ever seen any of your parents protest the idea of you taking your pacifier? Ask your parents if they would change their minds if you changed your mind.', '[title] Starting with simple changes, like not hoovering up bugs on the carpet but letting them out, are small yet effective ways of including pacifism into your life. [title] Try to read up on the jainist, buddhist or hindu principles of ahimsa.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}