{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f36ab436-f9a8-5819-bbef-3936a10a84af","task_key":"default--validation--50050","task_revision_id":"1","upstream_id":"50050","short_description":"HellaSwag validation 50050","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to cook a bone in ham [title] Determine the size of ham you need. [step] Generally bone-in roasts weigh 10 lbs. ( 4.5 kg) or more.\",\"endings\":\"['If you plan to cook whole ham without bones, add more or less meat. [substeps] Russet, white, and red ham are the best types of ham.', '[substeps] Most boneless or fatter roasts are between 10-16 lbs. ( 1.5 kg) or more.', \\\"Ham caramelizes greatly in its cooking process, therefore the size must be roughly the same. For this reason, most ham isn't as thick as the rest of the ham.\\\", 'They can feed a large family on a special occasion. [title] Pick an uncooked ham, if you want to cook it yourself.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}