{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ce289086-41bc-5822-8e4a-35944ac49917","task_key":"default--validation--47975","task_revision_id":"1","upstream_id":"47975","short_description":"HellaSwag validation 47975","config":"default","split":"validation","body":"{\"activity_label\":\"Wrapping presents\",\"ctx\":\"After that you put it in the center of your wrap, twist the two ends and get some ribbon to tie the ends. then you\",\"endings\":\"['can unfold the bottom of the tissue paper to make a flower look.', 'put the bottom of the finish with one bit of ribbon.', 'rub the wrapping paper and ribbons around the ends of the present.', 'fasten the four ones and wrap it in a wrapping paper, and put it on the hanger.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}