{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ecafdfe4-9b5e-55aa-83d8-39949f0a4da5","task_key":"default--validation--15532","task_revision_id":"1","upstream_id":"15532","short_description":"HellaSwag validation 15532","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to make a natural insecticide [title] Look for holes in the leaves. [step] There are different types of insects that can be problematic in the garden, including ones like caterpillars and beetles that chew holes in leaves. These types of bugs can cause a great deal of damage to plants, so you should take steps to get rid of them.\",\"endings\":\"['[substeps] Along with using insecticide to treat caterpillars and beetles in the garden, you can also remove them by hand. Pick the bugs off the plant with your fingers and drop them into a bucket filled with soapy water.', 'To locate the holes, feel around the edges of the leaves and pull them up and down. There should be tiny holes where the insects attach themselves, usually located at the base.', \\\"[substeps] Whether the leaves are crispy or look like they're teat-infested, insects like caterpillars are more likely to chew. Watch out for tiny vents in the leaves.\\\", '[substeps] Holes in the leaves will also often be associated with insects that grow inside plants. You will want to examine the small holes near the front and sides of the leaves as well.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}