{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ede6f438-03b9-56fa-929d-20bde82e99bd","task_key":"default--validation--43820","task_revision_id":"1","upstream_id":"43820","short_description":"HellaSwag validation 43820","config":"default","split":"validation","body":"{\"activity_label\":\"Grooming dog\",\"ctx\":\"Woman is with the black dog giving him a shower and styling his hair. Woman is talking while in the bottom a blonde woman is combing a white puddle. blonde woman\",\"endings\":\"['is mixing straws in a stainless steel sink as she show the signal to use, wag the tail, shake the head and end with a horseradish covered brush.', 'holds the white puddle and is talking to the camera giving the web adress to the dog salon white shows how comb and shower the dogs.', 'is getting her mane brushed by the woman in the bottom and combing the white away.', 'put the brown hair and turn the camera off.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}