{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2d5685b7-1639-5aeb-be4f-99ab7854551e","task_key":"default--validation--6280","task_revision_id":"1","upstream_id":"6280","short_description":"HellaSwag validation 6280","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to serve merlot wine [title] Ensure the correct temperature. [step] Temperature is one of the environmental factors that can greatly affect the taste of your wine. If it is too cold, you can hardly taste the flavors.\",\"endings\":\"['If it is too warm, the flavors get mixed together to create a tart wine. [substeps] You can adjust the temperature to 130 degrees fahrenheit (150 degrees celsius) or 105 degrees fahrenheit (49 degrees celsius).', 'When you are serving a wine using a bottle, check the temperature with a thermometer. [substeps] Most wine drinkers and servers should control the temperature of their wine to use southern temperate wine, but you should not have to do this.', 'If it is too warm, the flavors will not taste sharp and defined. [substeps] Merlot is best served at 60-65 degrees.', 'A good temperature for wine is about 70 degrees fahrenheit (148.1 degrees celsius). Too cold can cause the wine to taste bitter, and while sourdough is great for taste, flavors at a temperature below 65 degrees fahrenheit can be considered bitter.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}