{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9a50fccd-d4a8-56f9-b059-d7df0b870f3e","task_key":"default--validation--1839","task_revision_id":"1","upstream_id":"1839","short_description":"HellaSwag validation 1839","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress well [title] Focus on fit. [step] If clothes fit poorly, no matter how attractive or fashionable you will not look well-dressed. Tight clothes look cheap and make you look heavy.\",\"endings\":\"[\\\"Don't wear a dress that is too tight or that is comfortable for you to walk in, or you're likely to look cramped or exaggerated. [substeps] Choose cut that cuts down low without making your torso too high and keep your back straight through the cle.\\\", 'On the contrary, the fit of clothing is important. Try to let your clothes do all of the work for you.', 'Clothes that are too large will make you look sloppy. [substeps] Just because you can get into the clothing does not mean it necessarily fits properly.', '[substeps] Stay away from clothes with holes and stitches. When this happens you look a whole lot worse.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}