{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7bef12b8-b25f-5edf-aecf-868a733448b7","task_key":"default--validation--2910","task_revision_id":"1","upstream_id":"2910","short_description":"HellaSwag validation 2910","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to choose ductless range hoods [title] Assess your budget. [step] Your budget will constrain the level of functionality and appearance you can expect from a range hood. The primary budgetary factor of a range hood is usually its aesthetic.\",\"endings\":\"['Think back on when you first initially purchased your range hood. You probably were at least an apprentice to a shop owner.', 'The more material you can work with, the better. [substeps] The fourth consideration to assessing while choosing a range hood is expected durability.', 'Slick, contemporary stainless steel hoods typically sell for a much higher price than other models. Allocate your range hood budget based on the money spent on your other appliances.', 'Specific factors like availability, construction, and handling of the engine can impact how much the range hood will cost you. Take the following factors into account when deciding the ideal range hood that will provide the most mass utilization for your vehicle : [substeps] Your price range.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}