{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fc1a0544-81cb-5a36-81ed-c14b9ac72fb2","task_key":"default--validation--41999","task_revision_id":"1","upstream_id":"41999","short_description":"HellaSwag validation 41999","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to harmonize [title] Play 3rds and 5ths on a piano or keyboard. [step] A piano or keyboard will assist you in familiarizing and hearing certain notes. Play the 3rd of a c note on the piano by hitting the, then hitting the key that's two keys up from it.\",\"endings\":\"['While you play, move the keys back to their original positions. Keep moving the keys back and forth until you reach an upper-level c note.', 'This will use c as the average starting note. Play the 2nd of c on the piano by hitting the 2nd key.', 'Hit that key in key combinations, the e key in your right hand, and then the a key in your left. Then, hit the a key for each key with your fingers, which will then move you to the 3rd of the row.', 'The 5th is two more keys higher than the 3rd. You can play the original note and the 3rd or the 5th keys simultaneously to see what a harmony and melody sound like together.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}