{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0a0dd1e3-9595-55b6-ba01-e207965ef576","task_key":"default--validation--41301","task_revision_id":"1","upstream_id":"41301","short_description":"HellaSwag validation 41301","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to have a cool facebook profile [title] Go easy on the apps. [step] Facebook is a site to keep in touch with your friends, not to fool around with farmville all day. They may be tempting and addicting, but don't add too many, because they will be just a nuisance when it comes to notifications.\",\"endings\":\"[\\\"These apps don't quite bring you together, and remain a nuisance for too long. Here are some other useful dos and don'ts for facebook : [substeps] Facebook is not like a website; rather, it is a google post page.\\\", \\\"[substeps] Try to add only friends you really care about. Most facebook applications don't like broken messages.\\\", '[title] Watch your status-try not to put everything you do in there. [step] If there is something you want your friends to hear or something cool happened, put it there.', \\\"It's a fun and easy way to make new friends though! [title] Sync your facebook page to your facebook and twitter accounts. [step] There's too much trash! [title] Get more positive about your being cool and having a cool profile.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}