{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25f9739d-1a29-566d-b1f9-d6bdd2f1864c","task_key":"default--validation--31791","task_revision_id":"1","upstream_id":"31791","short_description":"HellaSwag validation 31791","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to dress your toddler in spring [title] Choose clothes your toddler will be comfortable in. [step] Make sure to consider your toddler's comfort when dressing her up for spring. Spring days are longer and tend to alternate between cold, warm and wet.\",\"endings\":\"['Choose outfits that will chill your toddler down before winter settles in. [substeps] Warm clothes like layers and waistcoats make for great warm clothing options for toddlers.', \\\"[substeps] Making sure to check the day's weather report will help you decide how best to dress your toddler. Make sure clothes are lightweight and cool, but that they can still protect your toddler from the direct exposure to sunlight when outdoors.\\\", \\\"Check the weather to make sure it's not too cold in your area. [substeps] Temperatures in the northern hemisphere are different.\\\", 'It is best to buy winter-friendly outfits for your toddler at the beginning of spring. [substeps] Just as the weather may be warm during the winter months, so too can temperature fluctuations during the spring.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}