{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2c0f88fa-d128-52d1-87d5-027486d15612","task_key":"default--validation--30945","task_revision_id":"1","upstream_id":"30945","short_description":"HellaSwag validation 30945","config":"default","split":"validation","body":"{\"activity_label\":\"Triple jump\",\"ctx\":\"A man in a black shirt is talking outside near a track and field. He then walks over to a student athlete from florida state university. the athlete\",\"endings\":\"['runs then jumps a ramp to make a long jump.', 'then begins talking and showing various techniques that she uses.', \\\"stands in the javelin thrower's position going up and down at his speed then flipping off the pole and landing on with his feet on the mat.\\\", 'then has several shakes and tosses multiple field hockey balls back and forth as the video pans back and forth between the two teams in the end.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}