{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"395c06f1-0974-54b2-a4d0-608a6096cf2b","task_key":"default--validation--36239","task_revision_id":"1","upstream_id":"36239","short_description":"HellaSwag validation 36239","config":"default","split":"validation","body":"{\"activity_label\":\"Making a cake\",\"ctx\":\"Workers are introduced as the efforts are displayed on how they are making a large upcoming contribution for a special event. the bakers\",\"endings\":\"['are then shown with their products, presentation followed by the group putting on decorations, packaging them, and installation them all together.', 'talk about the building they have bought for this event.', 'are then interviewed while prepped on a lawn.', 'are making a huge delicious cake for celebration of many years in special olympics.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}