{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0a983829-f6f8-5e0c-b48e-295f894584e0","task_key":"default--validation--48942","task_revision_id":"1","upstream_id":"48942","short_description":"HellaSwag validation 48942","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to live green and save some money while doing it [title] Use less electricity. [step] You can begin by thinking about when electricity is necessary or not. Try to use natural light every-when you can in order to save some electricity.\",\"endings\":\"['Find a fast-flowing fuel source at least three times a day or whenever you can. [substeps] The good news about rechargeable fuel is that it may be cheaper than gas and electricity depending on where you live and where you have neighbors.', 'For example, if you want to read, you can sit next to a window and use natural day light. [substeps] Turn off the light if you are not using it.', 'If there is no electricity in your living space, buy filters, condenser tubes, etc. Look for ones that are water-efficient.', 'Most of the time, grey areas need less than 100 watts in most places, which is why your living green is so far from trees, ponds, light sources, and buildings. [substeps] To purchase energy cards online, go to orange sporting goods websites, kickstones.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}