{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f801ae8-97d8-5473-a900-aa63c6dc3fd5","task_key":"default--validation--38660","task_revision_id":"1","upstream_id":"38660","short_description":"HellaSwag validation 38660","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to buy a summer wardrobe [title] Write a list of the basic items you need for your summer wardrobe. [step] For example : [substeps] Shorts and skirts some sleeveless/short-sleeved tops t-shirts dresses tank tops sandals and flip flops hats. [title] Get a dressy item.\",\"endings\":\"['[step] At minimum you need a tank top, a pair of shorts, and a pair of flip flops. It is best to have a few pairs of flip flops with you so you can measure your feet each day.', '[step] Whatever the season, a dressy item can be quite inexpensive. For example : [substeps] A sun hat a t-shirt a decent amount of knee-high/high top flip flops mini skirts an airbrushed or natural shape box.', \\\"[step] If you're going to a occasional event or somewhere that is more dressy than usual, then get some heels. Try to get heels that have some exposure for your toes, to allow for great ventilation when it's hot.\\\", '[step] For example: a pair of heels, tights, knee high boots, sandals or watch, a sash, and sports-themed accessories such as a ball cap. Boots can be a spring or knitted accessory.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}