{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c6082edd-51c7-5dbc-b02d-154aeb599a7a","task_key":"default--validation--41830","task_revision_id":"1","upstream_id":"41830","short_description":"HellaSwag validation 41830","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to solve the bikini bottom problem [title] Put on your bathing suit. [step] How much of your pubic hair shows? If it's just a small amount, it can't be too hard to get rid of. But if it's a lot you might need a little extra elbow grease.\",\"endings\":\"['The bikini size problem begins when you attempt to slide the bikini over your skin. Most bikini tops are designed to withstand the heat and humidity of the sun.', 'If you have hair-removal supplies, get it out. Otherwise, go shopping at your local department store.', 'You might want to get yourself a notebook so you can write your answer even faster. [title] Find a pair of shorts that can cover your bikini.', '[title] Remove the bikini bottom from its original wrapper and place it in the microwave. [step] Stir if needed to get a little grease on the bikini bottom.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}