{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"59bb314b-63e3-590c-b071-201dcf156e4c","task_key":"default--validation--47973","task_revision_id":"1","upstream_id":"47973","short_description":"HellaSwag validation 47973","config":"default","split":"validation","body":"{\"activity_label\":\"Wrapping presents\",\"ctx\":\"French cancan candy wrap, you need tissue paper, you fold it down really nicely. then you\",\"endings\":\"['brand a knife with the tissue paper to get it out of sight.', 'place the cellophane, you put white paper around the paper, you mark paper on it and the paper, and wipe it with tissue paper.', 'get scissors and you cut off some of it, you put the gift in a different piece of tissue paper and fold it up nicely.', 'push the package of candy nut cake through the flour to top it and next you put the chocolate over it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}