{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9b152393-7f16-5489-aa9c-490bda528660","task_key":"default--validation--17303","task_revision_id":"1","upstream_id":"17303","short_description":"HellaSwag validation 17303","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make pork roast with garlic and rosemary [title] Prepare your grill. [step] Using your favorite method light the grill. Have a bed of coals on one side of the grill, if you are using charcoal briquettes.\",\"endings\":\"['Keep in mind that we will be using what we use for this. Used coals will smoke better than burnt-out coals on an gas grill.', 'The coals will become the butt of the roast. Use glass or aluminum pans to hold the heat in the pan to keep it from burning the meat on one side.', '[title] Close the grill and head to the kitchen to prepare the pork roast. [title] Add the garlic and rosemary to a food processor bowl.', 'If not, you can place wooden skewers on the side opposite of the grill to line in the coals. [title] Pour your pork roast stock or water over your coals.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}