{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bfffb2b3-d6d7-5583-a2d9-a19c5ae1dde0","task_key":"default--validation--30638--variant-b6080db0535c","task_revision_id":"1","upstream_id":"30638--variant-b6080db0535c","short_description":"HellaSwag validation 30638","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to choose olive oil [title] Do not be deceived by location. [step] Not all olive oil is created equal. Usually, the bottom of a bottle of olive oil says where the oil is from.\",\"endings\":\"['Labeling can sometimes be intentionally deceitful. The company may boast their olive oil is from italy, but the oil may only have been packaged in italy.', 'That means the oil is far more concentrated than it is concentrated in the bottle. [title] Look for a dry place to store olive oil.', '[substeps] Look for olive oil in a branded selection, or find a consignment store where there are premixed olive oil brands. Visit a grocery store, drug store, or place where almost all olive oil is marked or identified by a \\\" label \\\" clearly written on the bottle.', \\\"However, it does, so you'll need to find out. If you're about to plant your herbs, then think about what shape you'd like your herbs to be in.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}