{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c82d681d-e6ab-5e59-ab54-300c74171b13","task_key":"default--validation--10353","task_revision_id":"1","upstream_id":"10353","short_description":"HellaSwag validation 10353","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to avoid common tax season scams [title] Be suspicious of threats. [step] The irs doesn't initiate communication with taxpayers, and won't threaten you with jail or a lawsuit. If you receive a call or email from someone who claims to be from the irs, it is an attempt to scam you.\",\"endings\":\"['[substeps] E-mails are a good source of data to put low-risk people on the hook. Messages sent from the irs can also be useful if you are facing out-of-pocket expenses for the tax year.', \\\"[substeps] For example, some of these scammers will threaten to suspend your driver's license. Driver's licenses are issued by state, not federal governments, and cannot be suspended by federal authorities for any reason.\\\", '[substeps] Predators often call victims at various angles and repeatedly try to motivate victims with threats of jail or a lawsuit. [title] Take action against an amusement park scam.', '[substeps] Be wary of single-party attempts to scam you or threaten to harm you. This includes inquiries with security inspectors that are directed at taxpayer-owned businesses.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}