{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e163defa-0f96-59bf-b3e8-4df9a3dab749","task_key":"default--validation--23466--variant-87c0b3270dd2","task_revision_id":"1","upstream_id":"23466--variant-87c0b3270dd2","short_description":"HellaSwag validation 23466","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to select shoes to wear with your wedding dress [title] Pick a shoe that fits your personality. [step] The era of traditional, high heeled, white, wedding shoes is over. Nowadays, you can get away with wearing any type of shoe to your wedding-be it dress shoes, sneakers, cowboy boots, or sandals.\",\"endings\":\"['Ask your friends, family, or coworkers to recommend something they think will feel comfortable for you. Make sure the shoe goes together well with your ensemble.', \\\"The most important thing is that whatever shoe you pick suits your personality. So you want to find one you'll feel completely comfortable and happy in.\\\", 'Consider what your personality lends itself to. [substeps] Go for a flat sole rather than a delicate one.', \\\"Instead, pick a style that works for the type of person you're in. [substeps] For example, if your short legs are appealing to you, then stilettos, booties, and pointe shoes work great.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}