{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6481913a-b1ee-5dfc-b9a1-cf27367fadac","task_key":"default--validation--32163","task_revision_id":"1","upstream_id":"32163","short_description":"HellaSwag validation 32163","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to deep clean a house [title] Clean your dishwasher. [step] Dishwashers can usually be cleaned by running a cleaning cycle. Before doing so, however, use a paper towel to remove any loose bits of food from the bottom of the dishwasher.\",\"endings\":\"['[title] Dump a good amount of water into the dishwasher. [step] Add either a drop of dish soap or a non-soapy dish detergent.', \\\"[substeps] Usually, to run a cleaning cycle, you place a cleaning solution in the dishwasher and then let it run. Refer to your dishwasher's specific instruction manual for the precise directions for your washer.\\\", '[substeps] If your dishwasher has a spray bottle, remove the top and spray the entire bottom with hot water. Once the water has completely cooled down, wipe any food residue from the top of the dishwasher with a soft cloth.', 'These can then dry in the same manner as paper towels. [substeps] The only difference this process makes is that without any food dripping from the bottom of the dishwasher, the dishes are likely to get dirty.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}