{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"65914cb2-f927-5efe-978b-b9d9aa0e0e93","task_key":"default--validation--1303","task_revision_id":"1","upstream_id":"1303","short_description":"HellaSwag validation 1303","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to be a scene kid without seeming fake [title] Learn what emo is, even if you aren't emo , [substeps] Wikipedia is great place to learn these things. Make sure you know where emo came from, and the original bands from when the scene first started. Doing this will make you prepared for when someone says you're emo.\",\"endings\":\"['A best way to reply is \\\" i\\\\'m not emo, that\\\\'s a type of music. \\\" check out some bands from the late 90\\\\'s through early 00\\\\'s.', \\\"Knowing how emo influences different people will help set you apart. Don't mistake emo ones for yourself.\\\", 'If you have many other parents living near you, make sure they are able to help you stay up-to-date on all of your activities. [title] Get to know the people you are surrounding you.', \\\"Don't let criticisms or frustration have a negative impact of your day. Find inspiration in patterns, events and situations that appeals to you, and combine them with your attitude.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}