{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4b559478-fb27-51c6-9200-a91a597bfa53","task_key":"default--validation--34754","task_revision_id":"1","upstream_id":"34754","short_description":"HellaSwag validation 34754","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to make an edward scissorhands costume [title] Create his scissor hands. [step] You can do this by unscrewing real scissors and attaching them to black gloves. Special effects artist stan winston and director tim burton used real, but custom made scissors in the film.\",\"endings\":\"[\\\"Make sure these are flat and square and don't have holes. The dimensions of the gloves are 56 inches.\\\", 'They were not as sharp or dangerous as they appeared to be and it might be a good idea an alternative material. Consider using plastic blades, cut out cardboard painted silver, covered in tinfoil or silver duct tape.', '[title] Hem the scissor hands piece to velvet. [title] Place the ripped body part on small table.', '[title] Make a sleeve to cover the hand. [step] Paint it pink, take a small sheet of white paper and cut a long rectangle.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}