{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"be2a6c05-c59a-5b4c-afae-32cda85bdbc1","task_key":"default--validation--18114","task_revision_id":"1","upstream_id":"18114","short_description":"HellaSwag validation 18114","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to survive a comet hitting earth [title] Pay attention to astronomers' predictions on any possible collisions with earth. [step] Nasa currently has astronomers that track comets that are flying in our solar system. As a result there's a very good chance they will know if there is a large object about to collide with us.\",\"endings\":\"[\\\"This can also be a factor that increases astronomers' odds that an earth is not going to collide with us. If you notice signs of a large objects approaching, coming to earth, you can expect the comet to collide with earth, as well as potentially an entire planet.\\\", 'By listening to their predictions you can develop a real sense of how much time you have to prepare. [substeps] Check back with the nasa website as they constantly update close calls.', '[substeps] Astronomers can also give advice on surrounding habitats, happenings, and events between the planets. After you witness an meteor fall or meteor strike, it is advisable to update your predictions on a regular basis and go over the safety procedures before deciding upon a course for your next event.', \\\"[substeps] Sometimes an asteroid can just drop our planet from the sky. When you hear a comet hitting in the sky, you know that you're in trouble! [title] Find ways to diversify your civilization if you want to maximize yourself.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}