{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0c62aed0-141d-5f84-9ed3-34c869679f91","task_key":"default--validation--44436","task_revision_id":"1","upstream_id":"44436","short_description":"HellaSwag validation 44436","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to pierce your eyebrow [title] Sterilize your equipment. [step] First, boil your needle for five to ten minutes on a stove. Keep your needle and your jewelry in rubbing alcohol.\",\"endings\":\"['Rinse your needles and jewelry thoroughly with cold water after sterilizing them. [title] Sprinkle the pick with oven-safe wax and let it cool.', 'When rinsing your needle, make sure your cutting edge is clean as well. Then shave your eyebrow with a razor and mild liquid sterilizing as suggested below.', 'Next, sterilize your threaded needle with hot, boiling water. Consider getting a small table sterilizer, which you can apply the goop to at a time.', 'If you have newly bought them, this might not be necessary, as they would already be sterile, but it is highly recommended that you sterilize them too, just to be on the safe side. [title] Locate where you want your eyebrow piercing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}