{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2dca317f-fada-5741-9efa-2f3ab9d6272f","task_key":"default--validation--38322","task_revision_id":"1","upstream_id":"38322","short_description":"HellaSwag validation 38322","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to epilate legs [title] Choose a good quality epilator. [step] The quality of your epilator matters. Pick a well-known, well-reviewed brand that you know you can trust.\",\"endings\":\"['The best brands that can reach you are probably revlon, trulia, and smith. [substeps] If possible, get an epilator that is ovulatory (even if there is a lower quality one like making ice cream with it).', 'You can find them in both consumer and automotive stores, and you should be able to find one at no charge. [substeps] A quality epilator should be made for comfortable handling and cleaning, and should not be expensive.', \\\"Do some research on the manufacturer and the types of epins used. Whether it's a branded epilator, or used worsted epilators, you'll want to start by reading reviews online.\\\", \\\"A cheap epilator may not work as well and it may be more painful to use. Since your epilator will last a long time (and you won't have to buy wax and fancy razors) it's worth springing for a good one.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}