{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b3bccb0b-ac61-5a39-bbe1-b7eb5c385208","task_key":"default--validation--7854","task_revision_id":"1","upstream_id":"7854","short_description":"HellaSwag validation 7854","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to create a bathroom shelving unit [title] Identify an area in your bathroom where you can hang the shelves and take measurements. [step] Visit the home improvement store and ask the lumber expert about types of wood that would be complimentary for your project, any types of wood on special and durability. Give the lumber expert your measurements and ask for advice on the amount if you are unsure how much wood you should purchase.\",\"endings\":\"['[title] Buy wood to create the shelving. [step] These shelves can be either stained or painted depending on your personal style.', '[substeps] Include a wall that you plan to hang your shelves on. This will likely consist of loose hanging boards and heavy boards for the shelves.', '[title] Make the room in which the shelves will be located. [step] Plan out the same area along the wall all the way around.', 'Remember, even a tiniest amount on one shelf can be a large lot of wood. [title] Decide what you will use for the shelving unit and perhaps some furniture pieces.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}