{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2a8b0b86-e110-5fa6-bc11-f9fa7ca85fbd","task_key":"default--validation--41807","task_revision_id":"1","upstream_id":"41807","short_description":"HellaSwag validation 41807","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"He finally finds the dog who is driving the riding lawn mower. According to the owner, the dog is asking for a raise. he\",\"endings\":\"['continues mowing in not very straight lines as the other dogs watch.', 'walks back to his lawn mower and starts to mow.', 'explains, and then let the owner poke holes holes in his lawn for the dog to go.', 'is raking the leaves in the wet lawn.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}