{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cc8c8178-db30-5263-975d-93a232dc7ab8","task_key":"default--validation--5613","task_revision_id":"1","upstream_id":"5613","short_description":"HellaSwag validation 5613","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to enjoy fashion [title] Take inventory of all the clothing you currently have. [step] Separate the clothes you like from the clothes you do not like. [title] Examine the clothing you like.\",\"endings\":\"[\\\"[step] If it's still too new or high maintenance, or if it is not as fashionable as you'd like, consider a different sport or new colour. Even if that means buying less, you might still find some essential features in the clothing.\\\", '[step] Look for similarities and ask yourself why you like each piece. You may like the way it flatters your shape, or maybe you like it because of the material or pattern.', '[step] Do you like the dress or dress length? Do you look great or look amazing? Is it a different style that might have a different function? If you do not like the dress, price your material at sales and reevaluate. If you like a dress but do not enjoy the dress, buy a suitable dress for wearing.', '[step] Is it a flattering item? Are they reasonably good? Be specific about what you like and do not like or dislike. [title] Make a list of all of your favorite styles.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}