{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fdecd878-b002-5ec7-9130-0eae07a3ca58","task_key":"default--validation--27392","task_revision_id":"1","upstream_id":"27392","short_description":"HellaSwag validation 27392","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make body oil [title] Find small, dark glass bottles to store your oil. [step] Sunlight and air break down oils over time. Store your oils in bottles made from dark, semi-opaque glass, the size of a small perfume bottle.\",\"endings\":\"['[substeps] You can also store your oil in a dark plastic storage container in your refrigerator. [title] Put your ingredients and oil together in your glass bottle.', 'The small size lets you work with quantities that you can use up before they break down, while minimizing headspace (air inside the bottle) that speeds up decay. [substeps] Use bottles with a tight cap or cork.', 'The fermentation process produces undissolved oils that dilute and react with other oils, like olive oil. Empty a shallow bowl of oil into another well-ventilated container and use it to store your oil.', 'This provides less warmth and moisture than an open bottle. [title] For powders, choose a brand or types of powders that contain 10% protein.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}