{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b99cfddb-e20b-5152-a046-1f956f916d3d","task_key":"default--validation--21071","task_revision_id":"1","upstream_id":"21071","short_description":"HellaSwag validation 21071","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to start a scrap metal business [title] Be able to identify and sort different metals. [step] Buyers will be purchasing specific metals by the pound. You will need to accurately identify your scrap for sale.\",\"endings\":\"['You\\\\'ll need specialized tools and equipment to perform this work. The first step is to identify which metals you have in your \\\" scrap.', \\\"To do this, first use a magnet to check the metal's ferromagnetism (whether or not the magnet sticks). Weigh this information with the metal's appearance, weight, and item of origin.\\\", 'You should have at least three types of metal for sale. Your scrap should identify you at this stage.', \\\"To start a scrap metal business, your first concern should be identifying which metal you are working with. The more familiar you become with how metals react with other metals, the more likely it is that you'll be happy with the result.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}