{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"068040ff-2462-5454-a271-6eaf97e284ee","task_key":"default--validation--211","task_revision_id":"1","upstream_id":"211","short_description":"HellaSwag validation 211","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to play five nights at freddy 's [title] Listen to the phone guy. [step] The phone guy was a previous security guard at freddy fazbear's pizza. He will give you useful and important information.\",\"endings\":\"['Some of the information he gives is to check security cameras often, as well as to close doors and turn on lights only if necessary. He also give some background information on the restaurant itself.', 'Here is what you need to know : [substeps] His real name is freddy fazbear chi, mostly because freddy fazbear signature and ghost are similar (actually, both). Freddy fazbear has a nickname that nobody knows.', 'In addition, he might recommend to your friend john for a concert on tuesday. This takes up the first week of leave period for your friend, killing him if he calls.', \\\"[title] Get to know the name of freddy fazbear's pizza. [step] By playing five nights at freddy fazbear's pizza, you can be able to play six nights at freddy fazbear's pizza.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}