{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a72be3b-675e-5a68-ab2e-e84862c19d7d","task_key":"default--validation--50368","task_revision_id":"1","upstream_id":"50368","short_description":"HellaSwag validation 50368","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to fix soggy rice [title] Let the water evaporate if there is water sitting in the pan. [step] Take the cover off of the pan to let the steam escape. Set the heat to low and continue to cook the rice for about 5 minutes.\",\"endings\":\"['[substeps] If you have a sieve attachment on your stove, perform this step. In most cases, the rice can drain into the sieve after about 5 minutes.', 'The water should be cooked off by the end. [title] Drain the extra water with a fine mesh sieve or colander.', '[substeps] You can also add 3-5 drops of tea tree oil or arrowroot oil to the pan at this point to help the rice cook faster. Make sure you add enough oil to cover the bottom of the pan.', 'The rice should seem to be bubbling when it reaches the proper temperature. [title] Cover the pan with an insulated lid.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}