{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"071b0ba7-78f3-523c-a5ba-c6a998ce38a4","task_key":"default--validation--6716","task_revision_id":"1","upstream_id":"6716","short_description":"HellaSwag validation 6716","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to become fashionable [title] Learn the basics of fashion. [step] You might not have an idea of what it means to be fashionable. If you're unsure what constitutes a fashionable style, spend some time learning about fashion itself.\",\"endings\":\"[\\\"You can do this by seeking out advice from friends and evaluating your own personality. [substeps] If you're unfamiliar with fashion, reading up on the latest styles might not be helpful.\\\", \\\"As an accessory collector, you'll learn about the reason people buy clothing. Whether it's it's getting a quote, overcoming the fact you fear poverty, or joining a red carpet event, learn what style values you find most important.\\\", \\\"First, it's better to be a fashion enthusiast than, say, a fashion photographer. If you have a good idea of some top trends, practice your technique.\\\", \\\"[substeps] Shopping at thrift stores usually isn't the best route; it might be tempting to incorporate trendy clothing into your wardrobe, but you'll be able to tailor your wardrobe to suit your current lifestyle. Try not to push yourself too far though.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}