{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a9cae27f-a314-53f1-bf7d-0f8abf3a04fc","task_key":"default--validation--11494","task_revision_id":"1","upstream_id":"11494","short_description":"HellaSwag validation 11494","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to spot early signs of cardiac arrest [title] Identify the signs of active cardiac arrest. [step] If you experience cardiac arrest, there is nothing you can do for yourself because you will be unconscious within seconds. You should be aware of the signs of an ongoing cardiac arrest episode and share it with those around you, so that everyone is prepared to act immediately.\",\"endings\":\"['[substeps] Someone who experiences cardiac arrest will collapse and become unresponsive almost immediately. She will not respond to taps on the shoulder or verbal commands.', '[substeps] Cardiac arrest occurs when the cells in the body store necessary oxygen to deliver blood to the heart into the bloodstream. The brain takes a break in sequence, but a problem occurs once that response is triggered.', 'If you are in an emergency and unable to relax, get to a doctor and inform them of your situation. [title] Confirm a \\\" cardiac arrest \\\" (ppp) qualification, which is a medical term for continuously making physical movements within a 30-second period of time.', 'Avoid withholding information or asking for information, and, if asked, avoid putting others in danger. [substeps] Look at the earlier stages of cardiac arrest.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}