{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7ab5257a-ee4c-564d-9bae-c921c801d2f2","task_key":"default--validation--20204","task_revision_id":"1","upstream_id":"20204","short_description":"HellaSwag validation 20204","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to score deer antlers [title] Air-dry your buck's antlers for 60 days or more. [title] Ensure the skull plate is intact and it has not been artificially replaced or repaired. [step] Your antlers can't receive an official boone and crockett score in this condition.\",\"endings\":\"['[title] Check the meat of the deer (outlined below) for defects that cause your antlers to be inedible and which could lead to a negative score. [step] Unfortunately, never foods red deer antlers, as this have food inside which animal have flaws.', '[title] Make sure your whitetail deer hunt follows the fair game laws of your state. [step] Unfair chase makes your antlers ineligible to be officially scored.', \\\"[title] Ensure the hooves have been fully dried before starting. [step] Grass dries to show and you don't want to give your deer an infection from the sweat of someone else hunting you.\\\", \\\"It must be old enough to be recycled properly by a store that sells crockett's, crockett's or crockett's antlers. As such, your antlers must be unblemished, so you cannot handle them if you can live without them.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}