{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a31dd34-cb6c-5e82-baf4-82b3eafaff10","task_key":"default--validation--38977--variant-2cb59f4364a4","task_revision_id":"1","upstream_id":"38977--variant-2cb59f4364a4","short_description":"HellaSwag validation 38977","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to be a sexy teenage guy [title] Choose your style. [step] If you need to, do some shopping. There are a few different styles for guys these days.\",\"endings\":\"[\\\"All are for girls, so do your research on each and every one before you invest in one, especially if you're new to it. Don't just buy all of the trendy styles.\\\", \\\"There are soccer shirts, but try to stock up. If you can't find one, that's perfectly fine.\\\", 'For examples:- [substeps] Boy shop: casual but sexy. Boy some period inspiration: edgy and glam; hippy chic girl: everything.', \\\"Go to a store you like and pick up shirts, hoodies, shorts, jeans, whatever you're comfortable in. And for the jeans: make sure they fit! If you really like the style and they fit in length but not waist, get a belt.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}