{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"81c474c6-36d4-5b7a-871a-23f52e3c7203","task_key":"default--validation--225","task_revision_id":"2","upstream_id":"225","short_description":"HellaSwag validation 225","config":"default","split":"validation","body":"{\"activity_label\":\"Having an ice cream\",\"ctx\":\"A woman is sitting at a table in a fast food restaurant while eating. She continually speaks to nobody as she eats. she\",\"endings\":\"['is then seen sitting alone as she eats a cold brown ice cream.', 'pauses in the process of eating to enjoy her food.', 'stands up and grabs her purse, continuing to talk and laugh as she leaves.', 'picks up another chocolate bar and chews it as she talks.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}