{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fd0afd93-4107-5674-b1f2-691d583c7858","task_key":"default--validation--26630","task_revision_id":"1","upstream_id":"26630","short_description":"HellaSwag validation 26630","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to light a road flare [title] Have a location in mind to place the flare before you light it. [step] It will generally have a rough striking surface on the outside edge. Remove the plastic lid or cap to expose this surface, if necessary.\",\"endings\":\"['[substeps] Light the strip at the opposite end of the road. [title] Direct power from the tube to the sputter wand (striker) for diameter.', '[title] Locate the cap at one end of the flare. [step] Often, the caps are designed with a fold-out tabs or a flat side that will help prevent the lit flare from rolling around.', \\\"[substeps] If the ground is hard, the place you will place your flare should be sheltered from wind. Small sparks are never allowed to enter a flare's ignition area, and you should never light flares in large open areas.\\\", '[substeps] A wooden container with a lid that is aluminized will do. [title] Search online for uv flares here.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}