{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f449f83-ecc3-5823-b605-75c2f0cc1409","task_key":"default--validation--1651","task_revision_id":"1","upstream_id":"1651","short_description":"HellaSwag validation 1651","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to ignore people you don't like [title] Stay away from the person. [step] Avoidance is sometimes the easiest way to ignore someone. If someone is getting on your nerves, try to keep your distance as much as you can.\",\"endings\":\"['[substeps] Walk away if someone enters your social circle. If that person waves at you and then leaves, stay where you are.', 'Try to stay away as much as possible. When you will be around this person, pretend you are not around them.', \\\"[substeps] You can avoid places they're likely to hang out. If an annoying co-worker always eats lunch at noon, try eating lunch outside of the office or taking a later lunch.\\\", \\\"If the person is annoying you and you didn't hear anything from them in a minute, don't even like them. [substeps] For example, don't visit a new shop when it's closed.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}