{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c213c91-3526-5e84-8408-bcc6a0910e15","task_key":"default--validation--45259","task_revision_id":"1","upstream_id":"45259","short_description":"HellaSwag validation 45259","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to freeze lemons [title] Place lemons in a zip lock bag. [step] Put your lemons into a zip lock bag. Zip the bag 3/4 of the way and force all excess air out of the bag.\",\"endings\":\"['Air will transfer quickly to the lemons. This method ensures that the lemons to freeze will stay fresh for 48 hours.', \\\"Once there's no air in the bag, zip it all the way. Forcing out the air will help keep your lemons fresh and free up space in your freezer.\\\", 'Zip the bag closed and place your lemon frozen lemons in the freezer. [substeps] When you begin freezing lemon frozen lemons in the freezer, they will have a bitter taste and will need to be blanched every 15 minutes.', '[title] Place the bag on the counter and leave them to freeze. [step] Allow to freeze overnight and then open the bag and remove the lemons.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}