{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9a35102a-f470-5ef5-ba78-eefeb85ca9dd","task_key":"default--validation--36240","task_revision_id":"1","upstream_id":"36240","short_description":"HellaSwag validation 36240","config":"default","split":"validation","body":"{\"activity_label\":\"Making a cake\",\"ctx\":\"Workers are introduced as the efforts are displayed on how they are making a large upcoming contribution for a special event. The bakers are making a huge delicious cake for celebration of many years in special olympics. many people\",\"endings\":\"['arrive for the event, and absolutely love the final outcome of the sweet cakes.', 'are in houses preparing for the event.', 'are still making the massive orts used by bakers per year.', 'are gathered in spontaneous events celebrating their accomplishments and their participation in the event.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}