{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ef5d55cc-8fea-5f3d-b32f-19f3e69ee55b","task_key":"default--validation--48797","task_revision_id":"1","upstream_id":"48797","short_description":"HellaSwag validation 48797","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make red eye gravy [title] Trim and slice the country ham. [step] Purchase about 6 ounces (160 g) of country ham and cut it into 1/4-inch thick slices. If you're using large center-cut pieces of ham, cut them in half so you have two medium-sized pieces.\",\"endings\":\"[\\\"[substeps] Note that some of the slices don't have sharp edges. You can make sure that they're evenly cut.\\\", 'Cut each slice in half on each side so the outside slices are roughly 1/4-inch pieces. [substeps] Some country ham recipes will require you to cut off the ham before making the gravy.', \\\"[substeps] If you don't have any pre-cut red eye gravy, you can eat the other breast of the ham before the gravy. [title] Heat the olive oil in a large skillet.\\\", 'Take a small knife and cut about 1/4-inch thick slits in the fat along the edges of the ham, about every three inches. This will prevent the ham from curling as you fry it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}