{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0548ecec-528c-5689-b91d-28867707bdd8","task_key":"default--validation--4645","task_revision_id":"1","upstream_id":"4645","short_description":"HellaSwag validation 4645","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear steel toe capped shoes in comfort [title] Assess the risk you have have from electrocution or heavy objects falling on your feet. [step] Not all employers require that you wear safety shoes on their site, however it is common sense to to consider the work you will be doing so that you can dress appropriately before starting the job. Don't forget to consider the whether or not there is danger of slipping and chemical burns.\",\"endings\":\"['[title] Find a pair that fits you well. [step] The risk of fire is at least greater when the pants are fitted properly.', '[substeps] In most cases, working from home and in a garage location, a painting will render and expense minimal. Employers may be concerned about the condition of older, newly placed equipment and materials.', 'Safety shoes are marked with symbols which show the type and level of protection they offer. Learn what the different safety symbols on shoes mean so you can choose the most appropriate pair.', '[substeps] A rust safety has to do with preventing the beginning of the fungus from spreading, which should be covered up immediately. [title] Alert the insurance company you are applying for.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}