{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"384fab20-9370-5539-811f-e2d27b7c25ea","task_key":"default--validation--4341","task_revision_id":"1","upstream_id":"4341","short_description":"HellaSwag validation 4341","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"A large mat is seen with a bar on top and two people watching on the sides. a man\",\"endings\":\"['is then seen running in from the side.', 'then joins the girl on the mat and begins hitting the red ball he is playing with.', 'then begins lifting the bar while still holding on to the other person and lifts the bar farther over his head and ends with the man in the end again lifting the bar up and throwing it to the parking lot.', 'is smiling while skiing down a hill, suddenly he stops and walks towards a house.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}