{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13cf2ef6-b320-598d-8766-d6a4c8496e59","task_key":"default--validation--43014","task_revision_id":"1","upstream_id":"43014","short_description":"HellaSwag validation 43014","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to recognize fear in dogs [title] Look for the whites in a dog's eyes. [step] A dog who's tense or fearful may widen its eyes so that they appear rounder than normal. A relaxed dog often squints so that the eyes become almond-shaped with hardly any whites showing at all.\",\"endings\":\"['[substeps] Some dogs may appear biting during relaxation or hiding from the thought of biting. The deweye dog flashes white teeth at the sight of blood, while another who exhibits an attacking psychological response may flinch or contort their faces into a fearful stare.', \\\"The whites of a dog's eyes may look milky, gray, or bloodshot, or very pale. [substeps] A calm dog generally relaxes when frightened or depressed, and may not retreat from fearful moments.\\\", '[substeps] Another characteristic of dogs that are tense or fearful is their pupils. The pupils normally dilate when they are calm or frightened.', \\\"If you can see lots of white around a dog's eyes, the dog may be afraid or tense. [substeps] Dilated pupils can also occur when a dog has its eyes widened and is another sign of fear or arousal.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}