{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eef4bb27-7e4d-5cc6-a2ef-1a0719de6e22","task_key":"default--validation--31168--variant-b4229f0625c3","task_revision_id":"1","upstream_id":"31168--variant-b4229f0625c3","short_description":"HellaSwag validation 31168","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to paint on canvas [title] Know the types. [step] To start painting on canvas, first thing you need is an easel. When you are buying an easel, you need to decide what the main function of your easel will be.\",\"endings\":\"['Also think about where you will do most of your painting. If you paint on the go, you need a small travel easel.', 'A good rule of thumb is to use a wide range of colors for your background colors. Bright colors can be awkward for the beginner, but think in a room with plenty of light, and a good lighting is key.', 'Some people think about using a simpler, more durable easel as the main focus. Others think of a more costly, easier-to-use canvas.', 'Take an overview of what canvas you are painting on to make sure you will have enough room and your easel can fit it all. Some painting styles even feature the \\\" skeleton \\\", a frame.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}