{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b7c1e717-4b02-58b4-ac3c-68b2e1c7446a","task_key":"default--validation--750","task_revision_id":"1","upstream_id":"750","short_description":"HellaSwag validation 750","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do 50s hairstyles for short hair [title] Do a side part. [step] To start creating a pin up style, brush your hair out like you normally would. Then, brush your hair into a side part.\",\"endings\":\"['Keep doing this step until you have all of your hair in a side part. [title] Keep your hair in this side part when you style it.', 'Half of the time you may need to comb your hair over to create less volume, and the other half of the time, you may need to hold your hair in a side part. [substeps] You can make sure you do both sides of your hair however desired.', 'You can part your hair on whichever side of your head is most flattering for you. [title] Section off the front of your hair.', \\\"Secure the hair to your head with bobby pins. [substeps] You can leave your hair down throughout the day or on the weekends so that it's just naturally part-down.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}