{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5040bb53-4be4-519a-a859-73edb9d9c4c1","task_key":"default--validation--10195","task_revision_id":"1","upstream_id":"10195","short_description":"HellaSwag validation 10195","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to buy a flat iron [title] Determine your hair type. [substeps] Consider whether your hair is fine or thick. Fine hair requires less heat to straighten.\",\"endings\":\"['Curly hair needs less curl in order to increase the volume, and heavy hair needs less curl in order to achieve a sleek, beach-like look. Generally speaking, medium, or thick hair should be naturally smooth and soft.', 'Thick hair is much harder to straighten because the individual hairs are coarser, causing them to have less moisture. Those with curly or wavy hair should purchase an iron with a higher heat setting.', 'Thick hair needs more dry heat so soft (but not very dry) hair will not straighten as quickly. Fine hair of all hair sizes, shapes, and hair types generally require a lot of straightening.', 'Thin people take less heat to get what they want because the hair fibers are the stronger. Thin people usually avoid blow drying because their hair comes much more naturally when it is dry and soft.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}