{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"28440d65-a05d-5227-bdcb-71d299247964","task_key":"default--validation--9552","task_revision_id":"1","upstream_id":"9552","short_description":"HellaSwag validation 9552","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to play volleyball [title] Choose your players. [step] Ideally, you want at least 6 players for a competitive game. However, if you're playing with a group of friends, keep the same number of players on each side.\",\"endings\":\"['It is also good to have 3 or more people to make game decisions. [substeps] The point of the game of volleyball is to quickly and smoothly guide the players around the court.', \\\"This way, you'll actually be able to toss through hoops together. [title] Determine who's responsible for shooting the ball.\\\", \\\"This can allow for more control, motion, and more dribbling opportunities. [substeps] When choosing players, it may help to consider who will play a particular area at different times of the game, though-don't split up your group of players.\\\", '[substeps] You should have 2 rows of players, with the first row closest to the net and the back row closest to the rear boundary of the court. The player in the back right corner will serve the ball.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}