{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9cc6e819-45af-53c2-a09a-3d5e75d3f0f1","task_key":"default--validation--20856","task_revision_id":"1","upstream_id":"20856","short_description":"HellaSwag validation 20856","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to remember long numbers [title] Look for shapes within your number. [step] Part of what makes numbers difficult to remember is that they are symbols that don't actually correspond in any way with the quantity of things they represent. You can get around this by trying to create mental images based on the shapes of the digits.\",\"endings\":\"['Your gray pieces (s) may not be perfect, and having many shapes can draw your mind more of the larger picture. [substeps] Think of shapes within a square or rectangle.', '[substeps] For example, if you have five digits, then five letters would be a good place to start focusing on on those numbers. [title] Say the numbers throughout your math problem.', '[substeps] For example, you might decide that the shape of the number 2 reminds you of a duck. You can then remember 2 as \\\" duck.', 'For example, your graph will not have an image of the various numbers used to represent things like \\\" sr \\\" with an image that is three times bigger than it actually is. [substeps] While that\\\\'s clearly in the shape of numbers, there are generally not any real information that can help you remember them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}