{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"87c67ac6-9412-5f10-9114-ba84265f3931","task_key":"default--validation--44732","task_revision_id":"1","upstream_id":"44732","short_description":"HellaSwag validation 44732","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to buy and store coconut [title] Find a place to buy coconuts. [step] This might be at a grocery store near you with a great produce section, or perhaps you know of a coconut tree nearby. Make sure that you have a good selection of coconuts to choose from.\",\"endings\":\"['Buying additional coconuts will greatly reduce the cost of moving a coconut to a special location. [substeps] Most types of coconuts come from either tropical forests or tropical jungles.', '[substeps] Although the best coconut season is from october to december, coconuts are being harvested all year long. [title] Choose a coconut that is heavy and full.', \\\"[substeps] Look online for a natural coconut store near you. You can ask for a cart to transfer the coconuts; however, if it's good to work with, ask the staff for the best selection for your needs.\\\", '[substeps] Consider asking for coconuts with their shell intact, so that you can add some of your own personal touches. Decide whether you want to wear the coconuts as jewelry or gift.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}