{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"03e0b504-6498-5047-95b3-0d0d2b3c405b","task_key":"default--validation--49271","task_revision_id":"1","upstream_id":"49271","short_description":"HellaSwag validation 49271","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to measure temperature [title] Use the thermometer to take body temperature. [step] While glass thermometers are not ideal for taking body temperature, you can still use them instead of an electric thermometer. Place them in the appropriate area of the body and wait between 2 and 4 minutes before reading the temperature.\",\"endings\":\"[\\\"[substeps] The ideal temperature to take is about 1 ° f (−32 ° c). If you were to place them in very hot water for 1-10 minutes, they wouldn't make a difference.\\\", '[substeps] Reading the temperature on an electric thermometer costs less than the average person with a normal body temperature of 19 ° f (- 7.9 ° c). [title] Use a thermometer to measure the temperature of your arms and legs.', \\\"Subtract your temperature from the temperature provided by a thermometer to determine your actual temperature. [substeps] As noted earlier, natural (and occasionally manufactured) thermometers are non-digital, so there's no need to visit an electronic thermometer in a digital format.\\\", '[substeps] Young children may bite and break the thermometer, which is why it is usually safer to take temperature using a digital thermometer. For children over 5 and adults, place the metal end under the tongue.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}