{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3b290985-b1e4-599e-a046-ed2492b45cde","task_key":"default--validation--10442--variant-636fe3856c1f","task_revision_id":"1","upstream_id":"10442--variant-636fe3856c1f","short_description":"HellaSwag validation 10442","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to shave your legs (male ) [title] Determine an end point. [step] Consider how much of your legs you wish to shave. Although this may not be true for everyone, men's legs tend to be just as hairy (if not hairier) the farther up they go, which makes knowing when to stop difficult.\",\"endings\":\"['[title] Decide if you want your legs to grow again to a point. [step] This cannot be answered by day and may involve trimming your legs if you plan to shave them again.', \\\"Think about the reason why you're shaving your legs: is it for aesthetics or a practical purpose? Then take a look at yourself naked in the mirror and decide where would be a suitable place to stop. [substeps] Consider how much of your legs and above will be visible to other people.\\\", 'An easy way to talk about your decision to shave is the time, place, and time. This will allow you to control your legs and make the best choice for your skin.', \\\"One key difference between men's legs is water marks. Water marks are those marks that fall off your leg beneath a shirt, while water marks are birthmarks.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}