{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"19d677e8-f194-5541-acfb-bb8ec2f08bbc","task_key":"default--validation--23862","task_revision_id":"1","upstream_id":"23862","short_description":"HellaSwag validation 23862","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to blacken steak [title] Choose new york strip steaks or rib eye steaks. [step] You want to use these lean cuts because they will cook evenly and blacken well. Choose steaks that are about 3/4-inch (1.9 cm) thick.\",\"endings\":\"['[title] Defrost the steaks completely in the refrigerator overnight. [step] Do not defrost them in the microwave, or you will ruin the process of blackening them.', 'Look at the internal temperature of the sirloin steaks and see if they are ready to be grilled. Use a grill thermometer to check for the internal temperature of the steak when it is on the grill.', \\\"If you don't want a table full, for instance, skip the steaks. [title] Sear the steak at 180 degrees.\\\", '[substeps] Before you cook the steaks, check each raw side. For thicker cuts, use 1/4 to 1/2 cup (1.7 to 1.75 g) of brisket for each steak.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}