{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b6646068-fe25-538a-bfd9-808080deec26","task_key":"default--validation--16637","task_revision_id":"1","upstream_id":"16637","short_description":"HellaSwag validation 16637","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be attractive in the fashion world [title] Work on your teeth. [step] If you're lucky enough to be blessed with perfect teeth, skip this part. If not, use it.\",\"endings\":\"['First, sort out any issues you may have with them. Visit your orthodontist, and see if you can get braces.', 'It all boils down to good oral hygiene. To get in shape, eat healthy, wear clean clothes and shoes, and smile often.', 'White teeth are not attractive in the fashion world. Find a smart, beautiful white-colored lipstick.', 'Eat a healthy diet that includes: fresh fresh-cut fruits and vegetables, whole grains, lean meat, lean fish, poultry, seafood, and legumes. Work on your teeth at least once a day.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}