{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b6c0bffe-ffa5-5bfb-90a4-c08f96a962f4","task_key":"default--validation--14116","task_revision_id":"1","upstream_id":"14116","short_description":"HellaSwag validation 14116","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to test for color blindness [title] Notice if you have trouble distinguishing between colors. [step] Children learn to assign certain colors to specific objects when they're young, such as by calling roses red and the grass green. If your vision is such that green grass and red flowers appear to be the same color, or closely-related colors, you may be suffering from color blindness.\",\"endings\":\"['[substeps] Color blindness usually occurs before puberty. During the adolescent years, a child may have an upper sensitivity to certain types of lighting, such as light sources and buildings.', '[substeps] Young children are likely to confuse the colors they see as well as any other colors they see. The only exception to that is when they realize that they see one other color in the same room.', '[substeps] Focus specifically on greens, reds, and oranges when analyzing your color perceptibility, as these are the most common colors affected by color blindness. [title] Keep a journal of your symptoms and concerns.', '[title] Use color correction in an environment that contains unusually dark places. [step] Objects that are highly reflective, painted, or are highly porous (like mirrors, cabinets, fans) could affect your ability to see colors through white.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}