{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"41367223-c26d-5a96-a983-f94d69888b7d","task_key":"default--validation--39414","task_revision_id":"1","upstream_id":"39414","short_description":"HellaSwag validation 39414","config":"default","split":"validation","body":"{\"activity_label\":\"Polishing shoes\",\"ctx\":\"He then shows a pair of worn down boots and a pair and a black pair of dress shoes. the man\",\"endings\":\"['stands in front of the small window with his shoes and smooths the pockets in the chair, then grips the long aluminum hanger.', 'touches the legs of the shoes to show the size they were used in.', 'puts on the shoes and the hot and cold foot irons he wore all over, afterwards he goes to the shower.', 'then picks up he black dress shoe and begin polishing it with a cloth, a stick, and brushing the shoes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}