{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b4ac0a17-e200-555f-a762-66672f7db91e","task_key":"default--validation--2233","task_revision_id":"1","upstream_id":"2233","short_description":"HellaSwag validation 2233","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to win an unemployment hearing [title] Read your determination notice. [step] If your unemployment benefits claim is denied, you will receive written notice of the denial. [substeps] The notice will include the reason or reasons your claim was denied, as well as information on how long you have to appeal that determination and the process for doing so.\",\"endings\":\"['You must also be aware of all the steps to remedy the situation and how others would want to handle things if you appealed. [title] Request that the statute of limitations be reduced.', \\\"Take note of the deadline for filing an appeal, and don't delay in requesting one. Depending on the state in which you filed your claim, you may have anywhere from 10 to 30 days from the date on your determination notice.\\\", 'For example, suppose your claim was denied because your employer \\\" alters a certain salary \\\" for tax purposes. You had that salary shortened to a lower salary at the time of that change.', \\\"You also may receive notice of your appeal if you didn't give concrete notice of your appeal. This notice also provides the amount of your penalties or other damages you may have to pay for any consequences you might receive.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}