{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7b8c5849-0b13-5bdf-a962-fd82492f7036","task_key":"default--validation--13617","task_revision_id":"1","upstream_id":"13617","short_description":"HellaSwag validation 13617","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to improve balance while riding a horse [title] Sit deeply. [step] Once you have mounted, you can start to find your balance. Sit in the lowest part of the saddle.\",\"endings\":\"['[substeps] Sit on the heel of a knee. Bend your middle and/or index fingers towards your stomach as you lift your leg.', \\\"Do not lean too far forward or back or your balance may be off. If the horse won't stand still, you can have another person hold the reins while you mount and find your balance.\\\", 'This is where your balance will come in. [substeps] Place one hand on the withers (the side section) of the saddle.', \\\"This is called the horse's give. Stand up and place your left hand on the saddle on the outside the horse's neck on the front side.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}