{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2d4c37dc-c4cc-55dd-baf2-b64410ddc668","task_key":"default--validation--1055","task_revision_id":"1","upstream_id":"1055","short_description":"HellaSwag validation 1055","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to save yourself from a heart attack [title] Watch for chest discomfort. [step] The principal sign of a heart attack is an uncomfortable feeling in your chest. It could feel like pressure is being put on your chest, that it is being squeezed, or that it feels particularly full.\",\"endings\":\"['It might go away only to come back shortly thereafter. [substeps] While we imagine that a heart attack comes on as an instant, intense pain, often it is a duller pain that slowly grows into a feeling of discomfort more than pain.', \\\"Your heart could be pounding all over your body, or in no particular location. See if your heart immediately begins to panic or kicks you in the chest, or feels like it's pressure poking or pulling at you.\\\", '[title] Take note of any shortness of breath. [step] The principal sign for a heart attack is considered a shortness of breath that is a symptom of a heart attack.', 'Severe chest discomfort could indicate a heart attack. [substeps] Increase heart rate by feeling uncomfortably heavy.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}