{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b62bfa93-7f63-5bd3-8d5c-728f4d442c18","task_key":"default--validation--35863","task_revision_id":"1","upstream_id":"35863","short_description":"HellaSwag validation 35863","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to make a flagpole [title] Cut a piece of pvc to your desired flagpole height. [step] For this flagpole, four to about seven feet high is best. Within that range, you are free to choose the exact height.\",\"endings\":\"[\\\"[title] Cut two lengths of pvc pipe to your desired height. [step] Use a friend's scissors, or any old fishing line that has been used to cut the pieces down to a length that you are comfortable using.\\\", 'Buy the pvc and either have the store cut it to length or measure and cut it yourself. Use a hacksaw to cut the pvc.', 'Line the pvc with a tape measure or kaxed plastic sheet before cutting. The jig here is simply to measure and mark the radius of each pole plus 12 inches (30.', 'It would be good to reserve a copy of your measurements for the sake of accuracy. [title] Orient a stick or dowel at the height you need.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}