{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"34079aa0-41bb-5992-9c7a-7fd1fe58dac6","task_key":"default--validation--6674","task_revision_id":"1","upstream_id":"6674","short_description":"HellaSwag validation 6674","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to measure your height by yourself [title] Gather the items that you will need to measure yourself. [step] Make sure that you have the following : [substeps] A measuring tape, a ruler or a yardstick a mirror a pencil a small box or thick book [title] Choose the right spot to measure yourself. [step] Pick a location that meets these requirements : [substeps] Find a flat, open space of floor next to a wall.\",\"endings\":\"[\\\"Select a high wall to brace for you. Have someone hold a yardstick close to your body to see if you're on the right side of the wall or above it.\\\", 'Lay down some newspaper and take a tape measure to the top of your head and on your forehead, chin, nose, and mouth. Measure from your jawline to the top of your ears.', 'Find a place where you can stand with your back to the wall. Find a place where you can make a small pencil mark on the wall.', 'Find a tabletop or table that stretches in length across the room. Make sure your support is sturdy and that your seat is not falling out of alignment or leaning to one side.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}