{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"66ae9b5b-1f9f-58bc-9adc-14af686bbf68","task_key":"default--validation--5214","task_revision_id":"1","upstream_id":"5214","short_description":"HellaSwag validation 5214","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to perform a no comply pressure flip on a skateboard [title] Get the feet correctly positioned. [step] Set up with your back foot off the tail of the board with just your big toe in the pocket. This is exactly the same position as the regular pressure flip.\",\"endings\":\"['Your feet should be perpendicular with the board, but farther apart. [title] Lift the toes of your board up.', 'This trick is almost all in the big toe of back foot, so this step is pretty important. [substeps] Your front foot can be wherever it is comfortable, perhaps just under the bolts at an angle.', \\\"[substeps] Make sure your ankle can't be crossed over that foot. You'll keep your hand gripping the tail.\\\", \\\"You may need a little more force to get the toe where it is. You'll know when to let go of the tail.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}