{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ed5bce78-2645-5070-8e5b-81bab8e6c612","task_key":"default--validation--15034","task_revision_id":"1","upstream_id":"15034","short_description":"HellaSwag validation 15034","config":"default","split":"validation","body":"{\"activity_label\":\"Rock-paper-scissors\",\"ctx\":\"They start to play paper scissors rock to see who will win. They have watchers standing by very stressed about how it will end up. they\",\"endings\":\"['just keep playing and one of the other characters starts to get really angry.', 'lose and start acting silly and throwing food at each other.', 'continue to play and keep the score in the end.', 'begin to play the game and go into a long huddle but a guy in black throws the ball and the players continue to play on the other side.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}