{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"080a61ba-0547-511c-9b88-19c0c09b2635","task_key":"default--validation--20815","task_revision_id":"1","upstream_id":"20815","short_description":"HellaSwag validation 20815","config":"default","split":"validation","body":"{\"activity_label\":\"Preparing pasta\",\"ctx\":\"Finally, the sauce is done and all of the ingredients are compiled and cheese is added to the dish as it is positioned on the plate. after\",\"endings\":\"[\\\", more entrees are shown and the screen ends on the url of ulampinoy's website.\\\", ', the pot is finally scooped out and the white house chocolate cake is finally cooked, cheese is added and the commentator begins his soothing news segment.', ', they put all of the ingredients into a boiling pot to give some flavor to the finished dish.', ', the sausage, mushrooms, chives are put into two glasses and placed onto black plates.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}