{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0678948c-99e1-59b4-b2d3-73b589e71d2e","task_key":"default--validation--24520","task_revision_id":"1","upstream_id":"24520","short_description":"HellaSwag validation 24520","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress in simple alternative fashion [title] Decide on a style. [step] You don't need to decide on a label; go with what catches your eye. However, just make sure you have a pretty good idea of what you want.\",\"endings\":\"['[title] Consider what \\\" texas style \\\" means to you. [step] Texas style is a standard style, but you should go with something more...straight-forward, elegant, fun, and versatile.', 'As always, try looking back in the mirror and removing any items that clash with your utilitarian look. You can refine your choices by researching online and by word-of-mouth.', 'It might help to take some extra time to think, and you might benefit from doing a shoe-browsing tutorial too. If your budget is high, a friend might be able to design.', 'Print or cut our pictures from the internet and magazines of styles you like to keep you inspired. Paste them to your wall, or create a little fashion notebook and put them in there, if you think it would look awesome with your décor.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}