{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e003a05-ae98-569a-bec2-fa52cefb413f","task_key":"default--validation--16379","task_revision_id":"1","upstream_id":"16379","short_description":"HellaSwag validation 16379","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose a watch [title] Determine why you want a watch. [step] There are hundreds of different types of watches available and they often have very different functions. A luxury watch that you will wear with suits or formal wear will be very different than an athletic watch you will use to time your runs.\",\"endings\":\"['If your experience is the sole object of running, opt for a watch that has both a physical component and a practical component. Well-respected watches include the firebrick watch, the rhd watch, and the mx watch.', 'When you are able to determine if this is a special occasion watch, or a watch for everyday use, you can begin to narrow your search. [substeps] Some common types of watches will be for athletic training, formal wear, everyday use, as a tech piece, or an antique piece.', \\\"Think about all of the different features and include any specific factors such as who the watch's owners are, including their names and trademark information. If you are looking for a necessity watch, then it is likely that one of these features would be readily available for you.\\\", 'A traditional watch may look very similar, but it is more expensive and worn on a necklace. [title] Choose a watch that suits your lifestyle.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}