{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7587ea3b-859c-547d-ac13-826f4b9d8654","task_key":"default--validation--45800","task_revision_id":"1","upstream_id":"45800","short_description":"HellaSwag validation 45800","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make mascara thinner [title] Add three to four drops of saline solution to dried-out mascara. [step] Use saline-based contact solution or eye wash. Be sure the eye drops aren't specially formulated to reduce redness, as this could actually irritate your eyes.\",\"endings\":\"[\\\"Pour a few drops in the tube, shake it, and then test the mascara on your wrist. Add another drop or two if it's still dry.\\\", 'Let the water sit on your eyes until the solution has sat on them for approximately half an hour, or half to an hour if you want to reduce redness. [substeps] Reduce the solution after each use if your hair gets dry.', '[title] Drip concealer powder onto your eyes. [step] The first ingredient in a second-aid kit is concealer.', '[title] Dab solution directly onto your lashes. [step] Add the mascara in you right up to the end of your lashes, or just above the eyelashes for a more dramatic look.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}