{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7cc7d611-02ff-5622-9a34-86d3be700099","task_key":"default--validation--3020","task_revision_id":"1","upstream_id":"3020","short_description":"HellaSwag validation 3020","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to plant brazilian waterweed [title] Keep the water's temperature between 72-78 degrees fahrenheit (22-26 celsius). [step] Temperature control is just as crucial for the plants in your tank as it is for the fish. Use a thermometer to regularly check the tank's water temperature.\",\"endings\":\"['The ideal temperature is 70 degrees fahrenheit (18 degrees celsius). [substeps] When the temperature is below this level, the plants will die and the plant will no longer have fresh water to grow.', \\\"If it falls outside the 72-78 range, you'll need to install heating or cooling systems as appropriate. Check your local pet store for guidance when acquiring these systems.\\\", 'If you see the temperature shift while watering your plant or the water has turned rapidly and the water feels too hot, it means it is too cold to plant. [substeps] Beans, carrots, squash, peas, and potatoes are all good choice plants to plant.', '[substeps] If the water is too hot, add 0.5-1% soy oil to keep it warm. Corn oil is optional, but should be added regularly as it is the least likely to damage your fish.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}