{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ff7b207-7f22-54e3-a43a-e67f5634f49c","task_key":"default--validation--43932","task_revision_id":"1","upstream_id":"43932","short_description":"HellaSwag validation 43932","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to complete poptropica wimpy boardwalk island [title] Go to wimpy boardwalk island and jump out of the blimp. [step] You need to click on greg, and you will see a short scene where money blows out of his hands and over to two teenagers. Rowley will ask you to help greg.\",\"endings\":\"['[title] He will let you go straight into the square, where you will find atrorude! [step] They will shoot you. [title] Follow atrorude and rowley.', '[title] On the beach, harry will ask greg to go over the water. [step] He will reply, \\\" ladies, can you go over this water to win my wallet? \\\" and then his body will explode into flame.', \\\"( isn't the color awesome? In wimpy wonderland island it's just black and white. ) [title] Go left to the area where there is a cotton candy stand.\\\", 'Promptly jump back onto the boat, through which the 2 teenagers can get into the jail. [title] Talk to greg and rowley outside of the jail.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}