{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cea8e282-a880-5ece-824f-08d68474d9b4","task_key":"default--validation--42401","task_revision_id":"1","upstream_id":"42401","short_description":"HellaSwag validation 42401","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to prevent calls from telemarketers (usa ) [title] Limit the amount of personal information that you provide to \\\" strangers \\\". [step] If someone asks for your name and address, they can easily find your phone number from the telephone directory. Protect your personal information as \\\" need-to-know \\\".\",\"endings\":\"['[substeps] Good personal information, like your name, job, political affiliation, social security number, etc, will prevent people from calling from your telephone or from your home. However, this is especially important if you have the phone number or address of people that you use to make calls.', 'For example, keep your personal information \\\" saved \\\". [title] Let old friends talk to you directly.', '[title] Do not call to respond to general media advertisements, television or newspaper ads. [step] The company can legitimately collect your number, through caller id, and may illegitimately share it with other companies.', \\\"Don't disclose where you live or phone number. [substeps] Most telemarketers will ask for your info, at least temporarily, before they attempt to track you down.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}