{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a1cb8196-e6a7-55e9-b1bd-414494dc0f5a","task_key":"default--validation--45177","task_revision_id":"1","upstream_id":"45177","short_description":"HellaSwag validation 45177","config":"default","split":"validation","body":"{\"activity_label\":\"Playing polo\",\"ctx\":\"The polo match begins and the men are riding on hroses that are going very fast as they all attempt to hit the ball with their sticks. there\",\"endings\":\"[\\\"are now six players on the field and they're all knocking three up on the field as they try to hit the ball out of the greens.\\\", 'are a lot of celebrities and even royalty at this polo event and every now and then they are shown, and sometimes they are also interviewed.', \\\"'s a laptop screen showing what she is looking at and with the line of men playing polo as well as the ball on the balls the men are playing.\\\", 'are also several jockeys lined up waiting to join the game and when they are finally able toyside they scatter and shake hands with some men standing at the front.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}