{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"950763f6-0247-56b2-9888-657e58c75cec","task_key":"default--validation--16621","task_revision_id":"1","upstream_id":"16621","short_description":"HellaSwag validation 16621","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"When the man reaches the pole he jumps very high, clears the pole, lands on the matting below it and people cheer while he gets up and runs back to the area he started. a slow motion replays\",\"endings\":\"['of this powerful turn of events is make while the judges give the boy specific instructions.', 'of a man running and beaching a pole is shown in slow motion in slow motion.', 'of the scene and the man smiles to the camera, pauses for a brief second then skiing down and skiing away.', 'and it shows the man running and jumping once again.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}