{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1acd6627-bc64-5f26-a6e4-ac44779c7dd8","task_key":"default--validation--46349","task_revision_id":"1","upstream_id":"46349","short_description":"HellaSwag validation 46349","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to freeze rhubarb [title] Trim off the leaves and tips. [step] Place the rhubarb stalks on a flat surface, like a cutting board. Trim off the top of the stalks where the leaves are.\",\"endings\":\"['Cut horizontally across the stalks into disks with a sharp knife. The disks do not need to be symmetrical.', 'These will be the reproductive pods; the seeds will be extracted later. [substeps] Avoid trimming the ends of the stalks because they will be dead, which is usually unhealthy.', 'Make sure you remove all the leaves, as these are poisonous. Trim off the bottom inch (2.5 cm) of the stalks as well, where the rhubarb was touching the ground.', \\\"Some rhubarb plants prefer cutting off the tips as opposed to the leaves. [substeps] If you're using branches instead of leaves, to keep the rhubarb roots healthy, use a plastic gardening knife to cut off the outer leaves from the stalks.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}