{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1f1ee4d5-56ea-5f10-9dfb-dfe79fc73682","task_key":"default--validation--49626","task_revision_id":"1","upstream_id":"49626","short_description":"HellaSwag validation 49626","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to replace a nixon 42 20 watch band strap [title] Prepare your supplies. [step] You will need the following : [substeps] Clean, dry, soft work surface. 1-2 small flat-head screwdrivers such as eyeglass screwdrivers.\",\"endings\":\"['( they will not make your watch beep, grinding it too much can damage your piece, so you want to make sure you have them handy. ) strainers to ensure that your watch falls out of place.', 'Suitable lubricant such as formaldehyde or iodine. A drawstring caplet (he may have a sticky side with buttons, you may wish to remove more).', 'Screwdrivers made for quick fix fixing televisions, and the like, since the switches are adjustable. Screws to unscrew latches on the dials, valves, and switches.', 'Replacement 20-21 mm watch band, for example, a shinola 20 mm black rubber strap. [title] Find the screw on either end of each lug (top and bottom).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}