{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b7843b78-6eb9-5600-b3f0-3ee79d15f7ac","task_key":"default--validation--20872","task_revision_id":"1","upstream_id":"20872","short_description":"HellaSwag validation 20872","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to break in patent leather shoes [title] Buy shoes that fit well. [step] Minimize the need for stretching or breaking in by getting patent leather shoes that are already a good fit for you. Since patent leather is stiff and unforgiving, you're not going to be able to change the fit very much-maybe a quarter of a shoe size, at most.\",\"endings\":\"[\\\"So, if you wear vans or hiking boots/flats everyday-you don't have to get in a calf-length patent leather pair. [substeps] Aim for a shoe that can comfortably sit any volume since highly laminate patent leather is durable.\\\", 'Have yourself fitted for shoes in the store, and try on a few pairs before you buy. Look out for the following when you try on patent leather shoes : [substeps] Make sure that your toes do not feel squeezed or crushed.', \\\"Thus, you want to make sure your shoes aren't comfortable when you go for regular walking or running shoes. Look for shoes with cuffs that are at least one-quarter a bit higher than your foot.\\\", \\\"[substeps] Don't wear cowboy-style shoes or high-heeled sandals. You'll weaken the patent leather and cause it to crack.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}