{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4dc38003-294a-57d2-ba5d-ccef6bbb1dc7","task_key":"default--validation--44408","task_revision_id":"1","upstream_id":"44408","short_description":"HellaSwag validation 44408","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to give yourself a tattoo [title] Buy a tattoo machine. [step] If you've never tattooed before, it's probably best to start with a tattoo machine, commonly referred to as a \\\" tattoo gun. \\\" these operate via electromagnetic coils, which control an armature bar, moving a grouping of needles up and down rapidly.\",\"endings\":\"['While popular in the united states, these guns run very smoothly and are relatively inexpensive. [substeps] Tattoos machines are required for almost any type of criminal activity, such as driving, fighting, or having sex.', 'They then light the entire area, reducing ink time and tingling sensation. You might find it helpful to have someone you trust teach you how to use the machine.', 'The needles are dipped in tattooing ink, which is applied under the skin. Tattoo starter kits with sterile supplies are available for around a hundred dollars.', 'The meter on the machine has an image of a gun printed in the upper left corner, in which a needle is attached to a line. Once the needle is pressed upon your skin, it switches into a present and a later, a picture of how your tattoo will appear.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}