{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eaf6a90e-c9a6-5f2c-a7ea-9462f9248e7a","task_key":"default--validation--20991","task_revision_id":"1","upstream_id":"20991","short_description":"HellaSwag validation 20991","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to lace boots [title] Select a boot lace. [step] Most likely you have a pair of boots that go at least as high as your ankles. Make sure that you get boot laces that are long enough for your pair of boots.\",\"endings\":\"['Most people who wear boots to work outdo boot laced or tailored boots. [substeps] Opt for a higher quality, more robust boot lace or something that is made for thicker boot laces, if possible.', '[substeps] Ask a sales clerk for advice or read the packaging to determine the right boot lace length. If you are getting new laces, measure the laces that came with your boots.', 'Pre-laced boots will not hold up properly if you want your boots to be as stretchy and tight as you want them to be. Take a pair of pre-laced boots, or go wide-trip and pick up two or three boots.', 'Remember, when lace is placed in the boot, it is more likely that it will slip up to make the fit that it needs for it to work. [substeps] Boots with a higher heel tend to be the easiest to lace, in this scenario, as boot laces will thread up the cuff of the boot instead of criss crossing, which cause more discomfort.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}