{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a22052af-d483-5ceb-881a-cf5242757054","task_key":"default--validation--18751--variant-9cc3089462c3","task_revision_id":"1","upstream_id":"18751--variant-9cc3089462c3","short_description":"HellaSwag validation 18751","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to choose a mattress [title] Know the different mattress options. [step] Mattresses range from classic firm or plush options to higher-tech mechanically-operated beds with remote controls. Do some research on these commonly available options : [substeps] Basic firm or plush mattresses.\",\"endings\":\"['These are more expensive compared to regular firmer mattresses, but cost less, use less foam, and can generally be found at used mattresses stores. Mattress models with dual mattresses also require additional storage space.', 'Most manufacturers will offer various mattresses to get them up and running. These models will allow you to learn the pros and cons of each mattress type.', 'These mattresses are more sturdy, will trap additional shock waves, and will keep your weight in balance and prevent calls from your medical provider. There are as few and as few mattress options as two beds.', 'Basic mattresses are made from a variety of materials and are sold at every price range. Mattresses made with organic or natural fibers tend to be more expensive.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}