{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1af7ed4e-e460-5525-abb3-c50e0919ed7c","task_key":"default--validation--20709","task_revision_id":"1","upstream_id":"20709","short_description":"HellaSwag validation 20709","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to childproof your car's interior [title] Place your child's car seat in the back seat of your car. [step] The back seat of your car is the safest location for your child in the event an accident occurs. [title] Position your child's car seat in the middle back seat.\",\"endings\":\"['[step] This can prevent your child from reaching or toying with the car locks and doors. [title] Verify that your child is securely fastened in the car seat.', \\\"[step] If your child is wearing a seat belt, you will place it in the middle of the back seat. It is the most natural place to put your child's car seat.\\\", '[step] The bottom edge of the car seats should face one another. [substeps] Try to put your child in the middle.', '[step] Choose a seat that is close to the left-hand side of the car where your child often chooses to park. You should designate this seat when you instruct your child to change locations in the car and to leave the back seat on the opposite side of the car.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}