{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c59e693-b382-5cc2-baae-8f41017534d7","task_key":"default--validation--28965","task_revision_id":"1","upstream_id":"28965","short_description":"HellaSwag validation 28965","config":"default","split":"validation","body":"{\"activity_label\":\"Welding\",\"ctx\":\"He cut the metal into half and bend the sheet to the camera, then he used a thong and clip the sheet of metal and securely put it in a holder. the man\",\"endings\":\"['is now onto a board and loosening his straps, after this he folds up the top portion and continues to lay the whole cake in on the piece of paper, then demonstrates how to tighten the edges by applying some metal and tightening and tightening the base to tighten all of the ends of the cake around the cake.', 'went to take a shower in the bathroom, afterwards proceeded to walk over and take a shower himself.', 'then switches the lighter to lighter and magnets on the end of the metal bar to create more power, then add a small baby blue flame tank and the camera then switches it back to a lighter lighter.', 'put needle on welding tool, put on his mask and began torching the metal, then poke it with hammer and show it to the camera.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}