{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e5bfe27-5cce-5c2f-8c08-ce815a9fb714","task_key":"default--validation--16729","task_revision_id":"1","upstream_id":"16729","short_description":"HellaSwag validation 16729","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to look cute in skate shoes girls [title] Find a shoe that you feel comfortable in. [step] What brand? Dc? Look through your closet. What is the main color? Black, white, pink, green, etc.\",\"endings\":\"[\\\"If you're wearing a red slip, seek a more neutral color. Black has a lot of feel to it, and in fact, we wear almost everything black! What color boots are you most comfortable in? [title] Look for some less-visible boots.\\\", 'You will find everything you need in there. If you are trying to attract girly girls you will need to look for great sneakers! After testing the shoes properly you can convince yourself to wear them.', 'What are the prices? [title] Know the basic basics. [step] That is, the shoes should have no removable heels and have no straps or buckles.', \\\"Do you like high tops or low tops? Buy a shoe that goes with your style and compliments the clothes that you already have. Don't buy a purple shoe if you don't have any purple shirts.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}