{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a7b3a0fc-bb7f-5de8-add1-9cadd02e563d","task_key":"default--validation--42318","task_revision_id":"1","upstream_id":"42318","short_description":"HellaSwag validation 42318","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to run a laundromat [title] Clean the laundromat. [step] Some light cleaning may be necessary, such as emptying lint from lint screens in the dryers, emptying trash cans and washing tables used by customers to fold laundry. If your laundromat has a lavatory, you will also need to keep that in good order.\",\"endings\":\"['[substeps] Wash all pre-washed and re-washed clothing according to season or day. Wash dirty clothing according to what season or the purpose of your laundromat.', '[substeps] Resetting the toilet by setting it in the appropriate position and then running water over it. If you also wish to move the wash table back to their normal position and remove the blind fold, wash that.', '[title] Perform regular maintenance on washers and dryers. [step] You will have to maintain the washer and dryer according to the type of machine.', \\\"[title] Go to the laundromat's waste disposal. [step] You can use the common household sink relocation system to clean especially dirty laundry.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}