{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3f1ee680-58c0-516c-990b-12ead62fa68d","task_key":"default--validation--7360","task_revision_id":"1","upstream_id":"7360","short_description":"HellaSwag validation 7360","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to start a saltwater reef tank [title] Pick the correct tank size for your fish and background. [step] You should determine the correct size tank by considering your level of experience, as well as the fish you want to keep. Research the tank size requirements for the specific fish and corals you want to keep.\",\"endings\":\"['Consider your level of experience with the hobby. Pick a tank size that you are comfortable maintaining and that is large enough for the organisms you want to keep.', 'These changes will be important when building a saltwater reef tank. [substeps] Saltwater plants should come up with their own space as you build them and live in it.', '[substeps] Fish like salmon, zebra, swordfish, or lake bass tend to need more depth to keep their populations up. Coral reef water seals well at around 1 , 500 lbs.', 'Do your research and purchase a tank you are reasonably sure is clean and sturdy. [substeps] If your tank is not sedimentary, and you are in a flood prone area, you need to take precautions.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}