{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"85b3cf3d-bfd0-51b6-bd6d-8b02a117719c","task_key":"default--validation--47974","task_revision_id":"1","upstream_id":"47974","short_description":"HellaSwag validation 47974","config":"default","split":"validation","body":"{\"activity_label\":\"Wrapping presents\",\"ctx\":\"Then you get scissors and you cut off some of it, you put the gift in a different piece of tissue paper and fold it up nicely. after that\",\"endings\":\"['you take the item out of the package and pick out some wrapping paper.', 'you put it in the center of your wrap, twist the two ends and get some ribbon to tie the ends.', \\\"you give it a small square box and it's addressed to the video store.\\\", ', you put the present together, spray some bubble wrap, put it in an oval holder with a straw, make sure it should be snug, remove the pink tissue paper, pin it on the cover, and then you put the tissue paper into a box and enjoy the gift.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}