{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ab90eba-53ba-537b-888b-63a8bc3e52ba","task_key":"default--validation--20541","task_revision_id":"1","upstream_id":"20541","short_description":"HellaSwag validation 20541","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to make upside down shepherd's pie [title] Heat up your skillet. [step] Place your skillet over medium heat. Allow to warm up for a few minutes before adding your ground beef.\",\"endings\":\"['As ground beef will leak grease when cooking, there is no need to add butter or oil. [title] Add your ground beef to a skillet and break it apart.', '[title] Carefully chop your beef up using a food processor or a sturdy bowl and three or four forks. [step] While you will be adding the ground beef, you want it to be smooth, lean, and even.', 'When the beef has heated up, add the butter and gradually melt it in your skillet. In about 5 minutes, the butter will melt and become flaky.', \\\"[substeps] If using ground beef, check the expiration date to be sure of the file number. If your cast iron skillet hasn't reached the proper temperature, you won't need to add ground beef.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}