{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9cf8afba-fcc0-5c19-8821-54c1cb21802a","task_key":"default--validation--38098","task_revision_id":"1","upstream_id":"38098","short_description":"HellaSwag validation 38098","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to run away from your stalker [title] Tell someone about the problem. [step] Parents, police or a trusted friend. Back your suspicions up with proof and other witnesses.\",\"endings\":\"['Always remember that nothing is guaranteed to be true, and sometimes, the best defense is to be outright honest with your stalker. Confide in them and make sure someone hears back from you as soon as possible.', 'Your best course of defense is to be as honest as possible. Why would you want to keep your secrets from somebody else? [title] Do not run.', 'As soon as you notice a stalker that is outside of your workplace or school, always call the police. The least that will happen is that they have a record of problem.', 'Some evidence can be circumstantial, some can be substantial. [substeps] Prepare evidence to refute the issue, e.g., a phone bill you accidentally pay.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}