{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"45c4a2b6-9dd9-500d-ade3-577387392b8a","task_key":"default--validation--17234--variant-0cab3c426622","task_revision_id":"1","upstream_id":"17234--variant-0cab3c426622","short_description":"HellaSwag validation 17234","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress up jeans [title] Make sure your jeans fit properly. [step] Choose a pair of jeans that flatters your figure and. Avoid jeans that are too snug in the hips and thighs.\",\"endings\":\"['For a dressy look, you should also avoid jeans that are loose and baggy. [title] Choose jeans which fit well in the waist.', \\\"You want to look like you're slim and graceful when you wear your jeans. Avoid baggy jeans that make you look like you're dead weight.\\\", 'Look for jeans that are flattering in the seams, areas of the leg with sleek lines, and tops with little to no detailing. [substeps] Avoid jeans made of any fabrics like silk, leather, or too thick and heavy fabrics, like cotton.', \\\"You don't want to dress up your jeans the wrong way. Stick to jeans made of shape-neutral or loose-fitting fabrics so that you don't show more skin than you already do.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}