{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"daf08422-9c3c-5607-9aec-677e64efecf1","task_key":"default--validation--2203","task_revision_id":"1","upstream_id":"2203","short_description":"HellaSwag validation 2203","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a bomber jacket [title] Consider fit. [step] Because they started as a utilitarian piece for servicemen strapped with equipment, they're naturally oversized. However, designers haven't been afraid to play around with the bomber jacket design.\",\"endings\":\"[\\\"You've got tons of options-from thick american blazers to ¾-1-come-out bomber jackets. You just need to try them on and see if one fits you perfectly.\\\", \\\"In fact, they've made many custom bomber jackets fit for military situations in the past. [substeps] Bomber jackets are wide-brimmed, functional, and tend to be less oversized.\\\", 'You can wear them as an oversized piece, but you can also find them in slim fitting versions, cropped versions, and so on. [substeps] Getting the proper fit is key to rocking this look.', 'Try a simple button-down and it can work great. You can use a plain button-down instead if you want to wear some seasonal \\\" steampunk \\\" materials.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}