{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"499c34e0-20c8-5635-b5c4-9317a1e25c37","task_key":"default--validation--25278","task_revision_id":"1","upstream_id":"25278","short_description":"HellaSwag validation 25278","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make cappuccino foam [title] Pour brewed espresso into your cappuccino cup. [step] Although specifics will depend on the type of cappuccino and the person making it, cappuccinos are roughly one quarter espresso and three quarters steamed milk. Pour your desired amount of brewed espresso into a large cappuccino cup before you steam the milk.\",\"endings\":\"['[substeps] Instead of cappuccino add straight milk. Espresso can be iced later, and most cappuccino companies encourage it to be iced after brewing to sweeten the milk.', \\\"That way, you'll have everything ready so you can pour out the steamed milk while it's still piping hot. [title] Fill a steam pitcher with cold milk.\\\", 'This will preserve your milk-stirring capacity throughout the rest of the process. [substeps] Try the recommended amount of coffee to get the right amount.', \\\"Start off with hot to ensure the coffee in your cup is sufficiently cold to immerse the foam. [substeps] It's best to use skim milk.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}