{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4c79c2f0-03a9-5b56-88ec-fc2a5142440f","task_key":"default--validation--14570","task_revision_id":"1","upstream_id":"14570","short_description":"HellaSwag validation 14570","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to make a burning laser [title] Select a diode strength and color. [step] The strength of a diode will be listed in microwatts (mw). The color of the diode will be determined by its wavelength (measured in nanometers, nm).\",\"endings\":\"['You can search online for diode strength and color charts. [substeps] Most lasers will be of 14 point, but you can also find lasers that are below 8 point.', \\\"[substeps] The direction of the lasers will show. The colors will indicate your lasers' heat element composition and the opacity of your lasers.\\\", 'You can find diode strength readings on the internet. Read the instructions on the dash website to calculate the strength of a diode.', 'A wavelength of 650 nm corresponds to a red laser, a wavelength of 405 nm corresponds to a blue laser, and a green laser has a wavelength around 520 nm. Green lasers are the most expensive, followed by blue.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}