{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2049d644-dd1b-5e4d-93e5-9b03dfd38e49","task_key":"default--validation--41833","task_revision_id":"1","upstream_id":"41833","short_description":"HellaSwag validation 41833","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to fit in [title] Identify the group that you want to fit in with. [step] Most likely this will be the popular \\\" in \\\"-crowd, but think of specific words that describe the group. This will help you better visualize and organize the things you will need to do to fit in.\",\"endings\":\"['Here are some common ones : [substeps] Dress up. Most social events will require you to dress up, so be sure to dress in layers.', 'For instance, if it is \\\" in \\\", \\\" fi \\\", or \\\" i\\\\'m in \\\", then consider moving on to the more formal ones. [title] Choose a group who fits in more with your current group.', \\\"[title] Consider the people that you'll attend. [step] This can include your friends, parents, new students or people that you might not have considered.\\\", '[substeps] Use this template: the popular kids are _ _ _ _ _ _ _ _ _ _. They are best described as _ _ _ _ _ _ _ _ _ _.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}