{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5730da2d-a606-5359-ac4e-4183dad5ceb3","task_key":"default--validation--25692","task_revision_id":"1","upstream_id":"25692","short_description":"HellaSwag validation 25692","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to identify simmental cattle [title] Note simmental cattle's weight and size. [step] Simmental cattle are quite large, and notably rugged in physical composition. They have a heavy bone structure, with visibly strong feet and legs.\",\"endings\":\"[\\\"[title] Look for simmental cattle's antlers and mane. [step] Simmental cattle are known for their shiny, pearly fur and headbands.\\\", '[substeps] Mature simmental cows will weigh from 1 , 100 to 1 , 500 pounds or more. Mature simmental bulls, which will show more muscle mass over their shoulders and hindquarters, usually weigh somewhere between 2 , 200 and 2 , 800 pounds.', 'Simmental cattle can stand around 7-8 pounds, and their bodies have such large hooves and powerful hind legs. Simmental cattle, however, have muscle that is often difficult to sustain.', 'Simmental cattle are more calcified than docile dogs. They can shed their fat without pain, and can learn to walk on grass, brush, dirt, and other hard surfaces.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}