{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b6fa7433-8d95-5c9c-8b03-70e2a7a859da","task_key":"default--validation--5175","task_revision_id":"1","upstream_id":"5175","short_description":"HellaSwag validation 5175","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to tile a countertop [title] Measure out your tiling plans in advance, getting all of your dimensions and marking problem areas like sinks. [step] Break down your counter into workable sections, dividing up the room by large obstacles like the sink or oven. Clean everything off the countertops and empty the kitchen so that you can move and work freely.\",\"endings\":\"[\\\"[substeps] If you don't own a tile mixer, you could also use an electric mixer with a wheel attachment attached. [title] Mark the types of tiles you plan to tile with the shortest segment of your method.\\\", '[substeps] Know your planned tile size, how many tiles you need total, and how the tiles will fit into rows ahead of time. If you can get a tile size that requires minimal cutting and fitting your life will be much easier.', \\\"[title] Lay out an entire wall on the floor for your working area. [step] This will ensure proper footprint that doesn't interfere with your work.\\\", \\\"[substeps] Don't leave two counters empty in a bathroom because of possible mold, mildew, or a cracked tile. [title] Estimate how much tiling supplies you will need to get the room you'll be working in organized.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}