{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5b33f291-c2ed-5297-911b-cf3b24d7cacd","task_key":"default--validation--22000","task_revision_id":"1","upstream_id":"22000","short_description":"HellaSwag validation 22000","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a fake lip ring [title] Get the wire. [step] Try your local craft store first (think beading stores), as they will likely have the best selection in terms of size and color. Large retail stores such as walmart will also carry a selection of craft wire.\",\"endings\":\"['[title] Take a piece of plywood and test a spot. [step] It should almost look like a pencil with a big sunflower attached on the bottom.', \\\"When you are at walmart you can also make your own for color, also, it's a good idea to get a wire. [title] Find the area where you will be practicing.\\\", 'They will be able to make a much fancier and identical kind of lip ring that is completely and totally fake. Just the holes in the wire are just going to make the ring look false to you and make it difficult to find it again.', '[substeps] Use wire in the 14-18 gauge range for fake lips rings. 18 gauge wire is thinner than 14 gauge and is easier to work with.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}