{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b9d7b100-5f73-5773-8a41-34f6f5f08947","task_key":"default--validation--4992","task_revision_id":"1","upstream_id":"4992","short_description":"HellaSwag validation 4992","config":"default","split":"validation","body":"{\"activity_label\":\"Playing beach volleyball\",\"ctx\":\"One of the guys hits the ball over to the other side and they hit it back. Then on the other side of the beach there is a group of women also playing volleyball. they\",\"endings\":\"['hit the ball on the other side and the team shots.', 'are bouncing and playing for fun while jumping up and down in the sand with their sticks.', 'serve the ball over to each other and they bowl over and around each other.', 'are playing a really good game, one of the girl lands right on her stomach.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}