{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"84896bbc-dc9b-594e-8323-a861e1637580","task_key":"default--validation--12113","task_revision_id":"1","upstream_id":"12113","short_description":"HellaSwag validation 12113","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to install hydraulic seals for various applications [title] Make working space. [step] Take out the hydraulic cylinder in an open space so that you can get enough space to work with. It will make it easier for you to install new hydraulic seal.\",\"endings\":\"['Make sure that the space is on a level surface. [substeps] If the open space is at the bottom of the cylinder, have a point of reference, like a rock, since you will be cutting into that surface.', '[title] Take out the parts of the older seal. [step] With the help of a seal pick, remove the older o-ring.', '[substeps] Also leave forklifts and any other heavy machinery out. [title] Prepare both slats and screws.', \\\"[title] Look for seal plates on hydraulic equipment. [step] The whole cylinder doesn't have to be completely spherical, as many carburetors and attachments have separate plates.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}