{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a0129263-73c6-564a-8f96-af68819f7962","task_key":"default--validation--11932","task_revision_id":"1","upstream_id":"11932","short_description":"HellaSwag validation 11932","config":"default","split":"validation","body":"{\"activity_label\":\"Drinking coffee\",\"ctx\":\"It is then mention with text on the screen that it can lead to high blood pressure it is then mentioned that it can hurt your hair growth and that it doesn't have any vital nutrients in it. it\",\"endings\":\"['gets finished quickly, then a video of a woman in a sweater is shown and then individual cut hair is displayed on the screen as she continues talking.', 'is then shown that there are benefits and that one should moderate their intake to keep a balance.', 'now is 3 pm and while the men are still brushing, one of them is getting interviews and what the man is wearing is funny and is even more funny.', 'shows large red bumps over the kale, which is some lettuce and lettuce, it has several plastic cleansers in it, and there is gum over there on the counter, and then it goes back to the video for another animation.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}