{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b1d26a5e-bd1c-54ed-ad33-076ed4a2b144","task_key":"default--validation--9003","task_revision_id":"1","upstream_id":"9003","short_description":"HellaSwag validation 9003","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to style a khaki dress [title] Choose a dress based on the weather. [step] Khaki can be worn in both the warmer and cooler months. When selecting a khaki dress to wear, keep the weather in mind.\",\"endings\":\"['Temperatures can skyrocket, especially in extreme climates, so keep an eye out for climates that will be frigid and rainy. If you intend to wear the dress in a place with higher temperatures, avoid top temperatures or long coats.', 'Is it okay for rainy days or overcast days? While variety is important, set expectations for your dress by creating clothes according to these factors. [title] Try a poofy skirt or a plain print.', \\\"Pick a dress that won't be too warm or too cold for the occasion. [substeps] For cooler months, try a knit khaki dress.\\\", \\\"The temperatures are usually very hot out, so avoid a dress that's too cold. [substeps] The hotter the weather, the more heat you'll burn off your khaki dress.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}