{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"49b38c81-4ed0-5800-a0a2-c7cd900bbefc","task_key":"default--validation--17638","task_revision_id":"1","upstream_id":"17638","short_description":"HellaSwag validation 17638","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to grow perennial vegetables [title] Grow jerusalem artichokes. [step] Jerusalem artichokes are a knobbly tuber that is usually prepared like a potato. [substeps] These can be very invasive so you may prefer to grow them in containers.\",\"endings\":\"['They do well in cooler climates and are best harvested after the first frost each year. [title] Consider growing globe artichokes.', 'Fill the tubers with young flowers, or seed them using a seed peeler. Seed them after planting for a rich, dried taste.', 'They also may fizz from being planted. They also produce a lot of insects that can cause the tubers to die.', 'Put the artichokes in a layer of water and clean the water by shaking it on the terrycloth. You can also try growing them outside in the sun but the temperature will rise after six months.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}