{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ef22024c-bfa7-58c4-8328-8e4045491a5e","task_key":"default--validation--8592","task_revision_id":"1","upstream_id":"8592","short_description":"HellaSwag validation 8592","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to cook diced beef [title] Remove the fat from your meat. [step] Use your kitchen knife to trim excess pieces of fat from the beef. Position your knife between the fat and meat.\",\"endings\":\"['As your trim the fat, keep it taut and almost parallel to the cutting board for better accuracy. Use clean, smooth strokes and start from the top of the steak.', 'After trimming the fat, use your hand to grab onto the beef at the thickest part. Pull the fat from the internal portions of the meat.', 'Hold the knife at a 45 degree angle and cut slowly as if you are cooking meat in slow motion. Keep your knife parallel to the ground throughout the cooking process to avoid jamming the blade.', 'From that point onward, the beef should be directly underneath the fat. [substeps] You can take off as much fat as you wish if desired.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}