{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e6fc5ccd-0a29-5ac5-9976-254882ca4ca0","task_key":"default--validation--28581","task_revision_id":"1","upstream_id":"28581","short_description":"HellaSwag validation 28581","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to build a concrete base in preparation for a garage [title] In order to make the shuttering easier and to ensure consistent thickness, prepare the ground and get it as flat as possible. [title] Assess the firmness of the ground. [step] If the ground is too soft, use hardcore to firm up.\",\"endings\":\"['In addition, make sure to clean the ground thoroughly with a sandblaster. If the ground feels spongy, use a weak sandblaster.', '[title] If you have soft ground, consider using a steel mesh to reinforce the concrete. [title] Install a damp proof membrane if the site where you are erecting the base is prone to damp.', 'If the ground is too firm, use crushed wood to support it. If the ground is too soft, use concrete to help stabilize it.', 'Hence, if the ground feels soft, use hardcore to stable the ground down. If the ground feels hard, use hardcore to place the concrete down.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}