{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9503be81-4567-5657-9bcb-6ec3e124d860","task_key":"default--validation--34076","task_revision_id":"1","upstream_id":"34076","short_description":"HellaSwag validation 34076","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"Then he shows another technique where he comes running and shoots the ball from very close to the hoop by jumping up high. he\",\"endings\":\"['throws the dart again and he misses it.', 'continues to show various other ways to shoot the basket as he runs and dunks the ball straight into the basket.', 'shows two other techniques similar but far more impressive as he climbs up to the hoop to perform.', 'also demonstrated how the hoop does a lateral jumping.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}