{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d83ad904-2df2-5beb-96be-956b67f6fafb","task_key":"default--validation--23574","task_revision_id":"1","upstream_id":"23574","short_description":"HellaSwag validation 23574","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to curve a soccer ball [title] Line up your kick. [step] Approach the ball at an angle. You will plant your non kicking foot about 12-16 inches away from the ball.\",\"endings\":\"['[substeps] Set up slightly off center from where you would if you wanted to kick the ball straight. A right footed kicker should approach the ball from left to right and a left footed kicker from right to left.', 'Then, hold your right (kicking) foot out in front of you, with your shin slightly bent and your right knee tilted to align your upper leg. Remember that the shins should be right under your front knee and your upper thigh should be held out behind your upper leg.', '[substeps] Also, position your dominant foot slightly forward and close. This will make it much easier to determine the right angle for the throwing motion.', \\\"This is the point where you're going to make contact. Make sure that your toe is pointing straight back, and that you've placed your weight correctly.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}