{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0a3f1e74-a3cb-5a8c-be86-094b29481bb2","task_key":"default--validation--47705","task_revision_id":"1","upstream_id":"47705","short_description":"HellaSwag validation 47705","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to play jenga [title] Set up the tower. [step] First, shake the jenga blocks out onto a flat surface. Then, stack the blocks in sets of three until you have built a tower that is 18 blocks high.\",\"endings\":\"[\\\"Have the crush (it's a good idea to have 8 blocks per player) at the upper-left corner of the tower to center a group of five. Have the crush (the person using the tower) standing behind the tower with three blocks of each kind the next person on the tower.\\\", 'Each new layer of three parallel blocks should be rotated 90 ° along the horizontal axis from the last layer. [substeps] Your jenga set should include 54 blocks.', 'Think of the tower as a basketball from the start. The tower should come to a 90 degree angle from you.', \\\"As you build your tower, go up and down several times, building blocks until the tower is completely built. You can even build a mini tower if you aren't building blocks yet.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}