{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"22ce13c7-822a-53c5-b29a-cb650b2a8eaf","task_key":"default--validation--22333","task_revision_id":"1","upstream_id":"22333","short_description":"HellaSwag validation 22333","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to reduce acne using tomatoes [title] Gather your supplies. [step] The most important thing is to make sure that you have tomatoes handy. You need to use fresh tomatoes, not canned.\",\"endings\":\"['[title] Cut about 10 inches (25.4 cm) of the tomatoes in half. [step] You will create a layer on your stove top.', 'Canned tomatoes are dark, tart, and salty. Use fresh tomatoes because they are hearty; tomatoes that have not been dried out can be very unhealthy.', '[substeps] Store leftover tomatoes in a resealable plastic baggie. Do not use canned tomatoes in markets, at outdoor fairs, grocery stores, or other places that tend to use raw vegetables.', 'Fresh tomatoes will not have added preservatives or seasonings, so they will be most effective for your skin. [substeps] Use full size tomatoes, not cherry or grape.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}