{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c8773122-994e-5334-91f9-e6e055fb8e90","task_key":"default--validation--5494--variant-06c589e0ac23","task_revision_id":"1","upstream_id":"5494--variant-06c589e0ac23","short_description":"HellaSwag validation 5494","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to perform a baseball slide [title] Sit on the ground and make a figure-4 with your legs. [step] Sit with your legs straight out in front of you. Fold one leg under the opposite knee, so you're forming a figure-4 with your legs.\",\"endings\":\"['Avoid crossing your legs, as this will make your legs more than a few inches apart. If you really want to move your legs, move your left leg forward and up toward the right leg at roughly a 45 degree angle.', 'Keep the toes of your forward foot pointing upwards. To choose which leg to bend, try both out and see which one feels most comfortable to you.', 'Your legs should almost be touching, almost parallel to the ground. [substeps] With each turn and step to your left, you should move your right leg to a 90 degree angle.', \\\"Don't try to cross your legs while you do this; this will create extra space between your legs. [substeps] Remember that sitting to your right will build up speed and the momentum necessary to turn the ball to your left foot.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}