{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c84cb498-1b04-5656-817e-9738632feaf6","task_key":"default--validation--49388","task_revision_id":"1","upstream_id":"49388","short_description":"HellaSwag validation 49388","config":"default","split":"validation","body":"{\"activity_label\":\"Ironing clothes\",\"ctx\":\"A man puts a cloth on a shirt on an iron board to iron, then the man puts the shirt on a hanger. then\",\"endings\":\"[', the man puts the shirt in a washing machine and after iron the shirt on the trunk.', ', the man uses ramps to iron the shirt, then he drapes a hanger on the ironing board.', ', the man talks and sets up the temperature of the iron.', ', he sharpen the knife to demonstrate how to lever the hanger, then he put the shirt on the iron rod.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}