{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9bfad507-74ff-54b7-af4a-3196cae7a0a0","task_key":"default--validation--49553","task_revision_id":"1","upstream_id":"49553","short_description":"HellaSwag validation 49553","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to protect your money [title] Exercise caution when using the internet. [step] Getting something for nothing is a hallmark of many of these scams. Virtually no one offers something for nothing, so if that is at the heart of a suspicious pitch, further investigation is necessary.\",\"endings\":\"[\\\"[substeps] If it sounds too good to be true, it almost certainly is. If someone claims they have an inheritance from a long lost relative, a high-paying-do-nothing job, or unclaimed lottery winnings from a ticket you don't remember buying, be very leery.\\\", \\\"If you suspect an open bill collector, tell them you're wary of using that bill. Ask them if there is a certain law against using these government websites and, if possible, how you can earn the money if a bill collector requests it.\\\", 'Resist the temptation to try to remove something for nothing, as this could make the signal to other people that you are not interested in the offering. [substeps] Studies have shown there are roughly 2000 per week people use the internet to confirm an item likes to satisfy their wants.', 'Anytime you do begin to use your internet for whatever purpose you send your fake email, it can result in you getting caught in the scam. [substeps] Do not allow any one to mistakenly read over your email.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}