{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"feb97cdc-1856-5c31-9fb9-4c42a095bab0","task_key":"default--validation--17680","task_revision_id":"1","upstream_id":"17680","short_description":"HellaSwag validation 17680","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to run a campaign stall [title] Anticipate and gauge interest. [step] This is the hardest part, but be sure you need a stall. If you're distributing less than either 50 magazines/50 slim newspapers/200 a4 or a5 leaflets, then you don't need a stall, you just need a decent bag.\",\"endings\":\"['Even an ordinary rucksack will generally hold this. [substeps] Choose a location where lots of people pass by and are likely to stop ie.', \\\"You can ask your friends to know (email or phone) how your week went. It doesn't mean they'll like it much, just give it a try.\\\", \\\"If you're distributing less than 50, you don't need a stall, you just need a decent bag. If you're distributing less than 50, you don't need a stall, you just need a decent bag.\\\", \\\"You should know it because sales going out will rise. So do not worry if it's not perfect.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}