{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9a0059fe-96d5-5f73-83d6-d9c31a774c6c","task_key":"default--validation--14019","task_revision_id":"1","upstream_id":"14019","short_description":"HellaSwag validation 14019","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to replace the screen on a screen door [title] Grasp the door on both sides firmly. [title] Lift it straight up inside its frame. [title] Tilt the bottom section of the door towards you.\",\"endings\":\"['[step] You will find you have to lift up both upside halves by adjusting the corner screw. [title] Place a rubber mallet or similar object between the outside of the door and the bottom edge of the frame.', '[title] On the side of the door, right above the screen door panel and to one side of the screen door, (where the lamp skimmer (or screen conditioner wand) holder is) is located, this is the glass panel. [step] There is usually a small glass panel panel near your screen door door.', '[title] Push the door down at an angle so the bottom comes toward your feet to free it from the frame. [title] Lay the door down flat on an even surface.', '[title] Place one thick-duty screwdriver (usually an adjustable thumbnail) in the corner of the screen, between the jamb and the door frame. [title] Push the other thick-duty screwdriver about halfway through, and hit your finger down firmly.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}