{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b4affe29-7585-5258-adfa-f3e34e61b840","task_key":"default--validation--13336","task_revision_id":"1","upstream_id":"13336","short_description":"HellaSwag validation 13336","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to buy tattoo flash art [title] Decide what kind of tattoo flash art to buy. [substeps] Select popular flash art styles. Choose the styles of tattoo flash art that are currently popular.\",\"endings\":\"['They vary in intensity and movement, and usually come from artists with specialized personalities. Widely-known artists include gray, blue, and diamond.', 'Some popular styles of tattoo flash art are tribal, celtic, maori and oriental. Choose flash art based on your customer demographics.', 'Think about what kinds of tattoo is sold and what cuts down you are comfortable with. Consider what style you could use for a large collection.', 'Consider different effects you are interested in using. Where is the sun coming from? Are there four windows? These patterns should be fairly similar.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}