{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"63b30a45-1c6f-58d4-ae4a-9759a19b996e","task_key":"default--validation--43611","task_revision_id":"1","upstream_id":"43611","short_description":"HellaSwag validation 43611","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to identify a fake watch [title] Listen for ticking. [step] This is one of the biggest indicators that a watch is a fake. A high quality designer watch is built with hundreds of small, perfectly crafted moving parts.\",\"endings\":\"['Pay attention to any clicking, creasing, or loosening of the watch, especially if you are buying it with other looks. Watch watches emit steady taps, which means they will sound unique whenever given a price.', 'Constant jiggling or rattling, and even continued shaking will tell you how many pieces are most likely fake. [substeps] Small parts are very noticeable or subtle while massive pieces that embellish all the things in a watch are interesting.', \\\"Therefore, it won't tick at all. To test this, hold the body of the watch against your ear and listen closely.\\\", 'Even if you hear it hitting the ground regularly, it may or may not chime, or emit a mechanical hum. Hence, a watch may have a fake watch on its neck.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}