{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d573b50-bc42-58fe-b087-a0cbd98a6b62","task_key":"default--validation--7731","task_revision_id":"1","upstream_id":"7731","short_description":"HellaSwag validation 7731","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to hire a bodyguard [title] Set a budget before you research possible bodyguards. [step] Depending on your needs, hiring a bodyguard can set you back from $200 to over $900 a day, with expenses depending on the level of potential danger and if you need an armed guard. Using your salary as a guide, decide how much you're willing to spend on a bodyguard.\",\"endings\":\"[\\\"[substeps] Bodyguards from respected agencies will cost more than less reliable bodyguards. Check many different bodyguard options to make sure you're getting the best guard for your price.\\\", \\\"[substeps] That's okay to buy a bodyguard who spends $600 a month on a later salary. However, think about how you would fit into the office or on a metropolitan area for 24 hours a week.\\\", '[substeps] Hiring a bodyguard can set you back $1 , 000 in the long run, but low pay will probably only attract more attention to you. Make sure to compare the potential bodyguards to determine the skills and personality of each one.', \\\"[substeps] Before deciding on a bodyguard, think about the commission you're willing to put on a guard. Hiring an armed guard will cost you a small fee, but a guard should even appear and tell you what fees they charge.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}