{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8ef6b600-d9b9-57e5-a94b-2dd33ea350b4","task_key":"default--validation--9731","task_revision_id":"1","upstream_id":"9731","short_description":"HellaSwag validation 9731","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to create an ecosystem in a bottle [title] Cut off the top of the bottle. [step] To make a bottle ecosystem, you will need an empty 2 quart (2 liter) plastic bottle. It is best to use a clear plastic so that you can see inside the bottle and make observations.\",\"endings\":\"['Cut off the top of the bottle, just below the neck. [substeps] To ensure your cut is straight, you can put a piece of masking tape around the bottle and then cut along the edge of the tape.', '[substeps] Use a pair of small scissors to trim any excess plastic from the bottom. If you want to create an ecosystem, this will appear the most natural.', 'Alternatively, you can use the top of a soda bottle. [substeps] Using a long knife, cut a slit into the bottom of the bottle.', '[substeps] Cut the top part off the bottle or just cut off any larger, rounded bits. The size of the bottle is totally up to you.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}