{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7278b40f-7d1e-5868-a1dc-f3478c15453d","task_key":"default--validation--3763","task_revision_id":"1","upstream_id":"3763","short_description":"HellaSwag validation 3763","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to remove iron on patches [title] Check to see if your item can take the heat. [step] Unless you added the patch yourself, you will first need to make sure your item won't be damaged by ironing it. Not all \\\" iron on \\\" patches are actually applied with heat.\",\"endings\":\"['Determine if your item can take the heat without removing the iron. [title] Remove any adhesive (if you have any).', '[substeps] The simplest way to tell if your item can take the heat is to check with a professional iron-on patch test kit. Over the counter formulas are known to require precision.', \\\"Patches available from home, the factory channel, and websites are the ones most commonly need to be ironed. Check your item's version of the iron, especially if you must use regular iron sheets.\\\", '[substeps] Pick a small, inconspicuous part of the item not easily visible during normal use. Place wax paper or a thin dish towel on top of the area.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}