{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2b4cc489-bc4e-5cb2-8443-6eeeb8c45055","task_key":"default--validation--8976","task_revision_id":"1","upstream_id":"8976","short_description":"HellaSwag validation 8976","config":"default","split":"validation","body":"{\"activity_label\":\"Removing curlers\",\"ctx\":\"This woman is showing viewers how to get big bouncy curls using hot rollers. First she lets the rollers heat up and she puts them onto her hair. next she flat\",\"endings\":\"['irons her hair and puts some rollers on it.', \\\"irons the bottom layer of her hair that doesn't have rollers on there and when the roolers cool off she takes them out.\\\", 'irons her hair and spikes it up.', 'irons the edge of the rollers for her hair.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}