{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"358c55af-d00e-5995-abfd-ac94b8fe1cea","task_key":"default--validation--49806","task_revision_id":"1","upstream_id":"49806","short_description":"HellaSwag validation 49806","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to make a fake arm cast [title] Use a washable marker to mark the cast lines. [step] Decide how far you want your cast to come down you arm. [substeps] For this type of cast, it's best to not go past the elbow.\",\"endings\":\"['This type of cast will look more like an older plaster-type cast rather than the fiberglass wrapped cast most often used today. [title] Wrap two layers of toilet paper around the bottom part of your arm.', '[title] Use tape to mark the cast lines. [substeps] Draw a line going down your arm, just from the elbow, and each line will repeat, similarly, on the opposite arm.', 'Mark an extra one inch down your arm to mark where the cast lines will be placed. [title] Use needles to mark the cast lines.', 'The cast lines come in two points: right and left. Your arm line should reach across your torso and stop when it hits the middle of your arm.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}