{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"0731d417-40f3-5c46-8cac-7a004713f8c5","task_key":"default--validation--1119","task_revision_id":"2","upstream_id":"1119","short_description":"HellaSwag validation 1119","config":"default","split":"validation","body":"{\"activity_label\":\"Surfing\",\"ctx\":\"The camera moves across the water and several people on boats and jet-skis are moving in the water near a large wave. More large waves begin to surface and a man begins surfing the waves. the forest\",\"endings\":\"['is then shown clearly under a large wide wave.', 'is then shown and surf boards appear that are washed up against the banks of the water.', 'starts to get darker and darker as the large wave picks up and men continue to sail around it.', 'of trees quickly moves over the trailing wave and they follow suit as well.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}