{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"763e9a68-fc96-5629-97c4-80119bbda168","task_key":"default--validation--32283","task_revision_id":"1","upstream_id":"32283","short_description":"HellaSwag validation 32283","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to use lock washers [title] Place the lock washer under the threaded fastener. [step] When used correctly, a lock washer will hold the nut or other threaded fastener in place. To help it accomplish this, put the lock washer on first, below the fastener.\",\"endings\":\"['It will do the job better if the fastener is positioned only below the threaded fastener. Make sure that you have three clips on the sides of the lock to open and close the lock.', 'If your project calls for other washers or hardware elements, they should go on before the lock washer so that it can hold them in place. [title] Make sure your washer is tight against the fastener and other surface.', '[title] Unscrew or loosen the nut or article that holds the lock. [step] Once the lock washer is loose, you can check the location of the screw or article.', 'This gives you more leverage and will help it to move. [title] Pry up through the top stopper of the lock.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}