{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f39b1cc-8267-5c89-bf7a-0183fd79ac87","task_key":"default--validation--35095","task_revision_id":"1","upstream_id":"35095","short_description":"HellaSwag validation 35095","config":"default","split":"validation","body":"{\"activity_label\":\"Swimming\",\"ctx\":\"Then a toddler get out the home, stand on the border of the swimming pool an falls in the water. the baby\",\"endings\":\"['is shown then a man gets a disc across the plastic gate.', 'floats face up, then a man comes and take the baby.', 'laughs when his friend visit, then he tether ropes and drinks water from the water.', 'lets go of the pole and runs, the man jumps in the water with an adult, and they are holding the puma.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}