{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ce88adf1-910a-52f9-baff-683305eb9478","task_key":"default--validation--41874","task_revision_id":"1","upstream_id":"41874","short_description":"HellaSwag validation 41874","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to calibrate a hydrometer [title] Identify the parts of the hydrometer. [step] A hydrometer is a glass device that has a bulbous, weighted end designed to float in a liquid and a narrow, long stem with a graduated scale on the other end. It is used to measure the specific gravity of a liquid.\",\"endings\":\"['[substeps] You can get a hydrometer that is labeled for gas both from the fermentation process, and from the reading of light and air. The rated gas for fuel is 44.99 °.', 'Specific gravity is the density of a liquid compared to water. [substeps] The bulbous end is placed into the liquid in question while the narrow stem will stick out of the liquid.', '[substeps] Hydrometers are often made of silver or stainless steel. When finished, the metal is much more solid, and will generally be made of real metal.', 'There are two brands of hydrometers available on the market, and the ones approved by international standards are : [substeps] Diichydrase (blue or grey without the label \\\" sodium hydrometer \\\"). The diichydrase meter measures the neptune\\\\'s mass.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}