{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"44609cc9-7ebc-53d1-a148-74c64488be71","task_key":"default--validation--6853","task_revision_id":"1","upstream_id":"6853","short_description":"HellaSwag validation 6853","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to hammer copper [title] Purchase a copper sheet appropriate to the specifications of your job. [substeps] Get approximately 2 feet (about 61 cm) more copper sheeting than you need for the job. Allow 1 foot (about 30. 5 cm) on each side.\",\"endings\":\"['Only purchase tools marked custom steel for the job. The iron plate will help greatly if you need to replace the plate of your choice.', 'Cut the sheets as soon as you arrive at the job site. [title] Get a surface on which to hammer galvanized sheets.', 'Purchase 2 inches (5.08 cm) pieces and use a calculator to find out how much copper will be needed for the job. Multiply the original amount from your allowance for the job to get the copper sheet.', \\\"It will shrink with extensive hammering, and you'll need the excess on the ends for hammering in nails. [title] Choose a soft temper of copper sheet to allow you to hammer copper more easily.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}