{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5859ed10-4fa3-5b0e-b02d-fa3afb5de180","task_key":"default--validation--7801","task_revision_id":"1","upstream_id":"7801","short_description":"HellaSwag validation 7801","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to choose accent pillows [title] Choose pillows that fit the area. [step] If you are putting accent pillows on a sofa that is a typical size, around 16 \\\"-18 \\\" x 16 \\\"-18 \\\", standard sized pillows will fit neatly in the space. Standard square accent pillows are around 18 inches.\",\"endings\":\"['Make sure you decide which size you will have the best base for your furniture since heavier pillows will require a different base. [title] Get drapes or curtains that block light.', 'However, if you are dividing it by 2 \\\", the pillows may be too big for the sofa. If so, go with 19 inch square pillows.', '[title] Cover your sofa with a fabric lining for a classic look. [step] Opt for patterned fabric or absorbent bedding if you want your accent pillows more neutral and quiet.', 'Having average sized pillows on your sofa will keep the area comfy looking but not overwhelmed by the pillows. [substeps] If you have a larger sofa or a larger bed, go for oversized square pillows, which are around 24 inches.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}