{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4c37a1e9-530d-512c-b317-86b4dc5a262f","task_key":"default--validation--27867","task_revision_id":"1","upstream_id":"27867","short_description":"HellaSwag validation 27867","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to attend an indonesian traditional wedding [title] Wear formal dress. [step] Females can wear almost anything proper, but pick the dress that is not revealing too much skin. Revealing your cleavage would be considered rude.\",\"endings\":\"['However, showing cleavage would be very embarrassing if other people, including your boyfriend or significant other, could see it, and would likely embarrass you. [substeps] As always, always have a reservation if there is no other option.', 'For males, it is a suggestion to wear batik and formal long pants. T-shirt and anything jeans is a no.', 'Also in the event of a recent makeover, you should wear skirts with a modest neckline without looking trashy. This will make your outfit formal, yet still appropriate.', \\\"You don't have to wear anything covering your breasts, as long as it's not showing as you are wearing your lingerie. Opt for evening wear, as that is usually considered more formal.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}