{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f3714ab6-b289-5c6f-8d1e-7467b3c747fb","task_key":"default--validation--8025","task_revision_id":"1","upstream_id":"8025","short_description":"HellaSwag validation 8025","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a fork bracelet [title] Choose your fork. [step] Look to buy forks at garage sales or thrift stores. Usually, silverware sold at garage sales and thrift stores are relatively cheap.\",\"endings\":\"[\\\"The fork that you buy also needs to be made of silver or sterling silver. That's because silver and sterling silver is very easy to manipulate and bend.\\\", \\\"If you're not sure what you're putting together, ask your cashier. [substeps] Check if the fork has 2 sides: the wider the better.\\\", \\\"However, having a fork for later use is a good idea. If you'll be using the forks multiple times, ask your friends to help you find forks.\\\", 'You can use your fork to make smaller, fancier rings, like the ones sold at thrift stores. [substeps] Choose a fork made of stainless steel.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}