{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"37e40246-ed56-51e0-bffa-9280383b3528","task_key":"default--validation--14530","task_revision_id":"1","upstream_id":"14530","short_description":"HellaSwag validation 14530","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to make a go bag [title] Purchase a heavy-duty duffel bag. [step] For your go bag, you want a bag that is large enough to carry the necessary items but not be filled to the brim. It should also be sturdy enough to carry all of the items without ripping or getting damaged in some way.\",\"endings\":\"['[substeps] If you have a backpack that you are no longer using, this can work. You might even want to consider getting a solar backpack that can charge various devices.', '[substeps] If you are planning to book a hotel, a heavy-duty bag will be less standard. Heavier bags are best on hotels or vacationing destinations, where furnishings are difficult to replace so you need as many items as possible.', '[substeps] If the bag is not large enough to fit your entire bag, you may want to consider purchasing a more heavy-duty bag. Look for the heaviest cases available at most major discount stores, such as target or walmart.', '[substeps] Try to buy a bag on wheels. [substeps] If possible, use wheels made from rubber.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}