{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0a8296e5-f33b-5052-9c71-d43b533ac208","task_key":"default--validation--4737","task_revision_id":"1","upstream_id":"4737","short_description":"HellaSwag validation 4737","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to conserve ammo in resident evil [title] Master the knife. [substeps] If you are less than proficient with the knife, there is still a way for you to use it, but will still require some ammo. Shoot zombies a few times until they fall, but not die.\",\"endings\":\"['Go over and slash their fallen bodies as quickly as possible. If they get up before you are done, finish them off with a gun.', 'If you are at a dead end (conserving low damage) try reinstalling your secondary and switch tactics. [title] Access the stamina hole (the one you know where all the zombies surround you except you).', '[title] Shoot someone until they fall, especially if you know they will try to retaliate against you, or die. [step] You need to kill them first, so go around town and see if anyone is.', 'Congratulate yourself when you win, and accept the reward. The only way you can live with empty ammo is using both knives, but its possible to recharge them without changing the swap method.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}