{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a30acce-6b8b-5db7-af17-78767d9af476","task_key":"default--validation--19416","task_revision_id":"1","upstream_id":"19416","short_description":"HellaSwag validation 19416","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to avoid losing baby shoes [title] Measure your baby's feet. [step] You can do this yourself or you can take your baby to a shoe salesman for help. Your baby's shoes will be less likely to fall off if they fit properly.\",\"endings\":\"[\\\"[substeps] Whether you take shoes off or put them on your baby, try to keep a close eye on each shoe's size. When actually measuring your baby's foot, you should measure 5 foot (1.5 m).\\\", \\\"Here is an easy way to measure your baby's feet : [substeps] Stand your baby up on a piece of paper and trace their foot with a marker. Measure the tracing from the heel to the longest toe.\\\", \\\"[substeps] Always check with the salesperson if you are uncertain about how big your baby's feet are. Remember, shop only one size up, and avoid buying shoes you don't think your baby will enjoy.\\\", \\\"[substeps] Start by measuring your baby's feet, not her toes. If your baby's feet are big, measure them from head to toe.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}