{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"44d73a9f-8d56-5695-b73f-1e4a12892804","task_key":"default--validation--11068","task_revision_id":"1","upstream_id":"11068","short_description":"HellaSwag validation 11068","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make breakfast in a biscuit [title] Cut the sausage into chunks using a sharp knife, and place the chunks onto a plate. [title] Preheat the oven to 400ºf/204ºc. [title] On a lightly floured board, roll out each round of biscuit dough into 4 inch (10 cm) flat circles. [title] Take the circles of dough, and put them in the muffin tin.\",\"endings\":\"['[title] While cooking, clean the length of the roll-out, and set it aside. [title] Separate the biscuit dough into different parts.', '[title] Scrape out any excess dough, using a pastry brush or your fingers. [title] Place 1 tablespoon of butter on a plate.', '[step] Look at the picture to guide you. Basically, place the dough in each hole of the muffin tin, and leave it open so that you can put the remaining ingredients into each dough circle.', '[title] Roll out each circle of dough in the same way, except this time with 8 large, round biscuits. [title] Place each biscuit in the centre of the muffin tin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}