{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6928eb51-e66a-5d19-b709-c4470e6e0f9e","task_key":"default--validation--31926","task_revision_id":"1","upstream_id":"31926","short_description":"HellaSwag validation 31926","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to do simple chromatography [title] Make or buy a chromatography strip. [step] A chromatography strip is a strip of material that the mobile phase mixture, a liquid or gas, moves through. The strip serves as the stationary stage so you can see the rate of travel of the different components in the mobile phase.\",\"endings\":\"['For this reason, you should learn how to purchase a chromatography strip. The chromatography strip is made of stretchy material that is safe for moving around the mobile phase components.', 'You can by specially formulated chromatography strips for superior results, but for simple chromatography you can make your own. Cut a coffee filter into straight strips about 1 inch (2.5 cm) wide and five or six inches long.', '[substeps] For this strip you will need plastic sheeting. Most plastic sheeting is pretty thin because it does not allow air to get through.', 'Some commercially produced medical devices have been designed to be accompanied by a chromatography strip itself. [substeps] A chromatography strip can also be used to draw up a model on a scrap piece of clothing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}