{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"750e7c76-d60b-5fea-8a2d-dc5aae02de1f","task_key":"default--validation--10923","task_revision_id":"1","upstream_id":"10923","short_description":"HellaSwag validation 10923","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to understand how a bill becomes a law [title] Learn a little background information first. [step] In the legislative branch, there are two groups. One is the house of representatives, and one is the senate.\",\"endings\":\"[\\\"Both groups rely on someone's voluntary vote to report how a bill is formed. The house of representatives will have the power to approve and will apply your bill throughout the law.\\\", 'Both are part of the legislative branch, with each group containing a certain number of people (the number is different for each state. ) these two groups in the legislative branch will be referred as the two chambers of congress throughout this article.', 'The house of representatives votes on behalf of a majority by voting on committees and resolutions. In this group, the person taking power of the house is considered the president.', 'Some committees are specifically created to retake mortgage over time, while others are created to foster voting. The house of representatives can be historically named entirely as a capital entity, with small goals coming up.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}