{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"79113d24-0da1-5556-972c-9e1027abf09c","task_key":"default--validation--36400","task_revision_id":"1","upstream_id":"36400","short_description":"HellaSwag validation 36400","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to identify bed bug bites [title] Inspect your bites. [step] Look for slightly discolored, pin-like raised red dots that are about 0.2 cm to 0.5 cm in diameter. You may also see hives or wheals that are significantly redder than the surrounding skin.\",\"endings\":\"[\\\"If you're experiencing a more rare, severe case, you might even find blisters larger than 0.5 cm in diameter that have developed on bed bug bites. [substeps] 1 centimeter equals 0.4 inches.\\\", '[substeps] Wax lice are stage 1 insects that can grow quickly. They can require you to feed them insects for an extended period, but are generally not harmful.', 'If the bites seem to be growing or becoming larger, they may be a sign of a head bug. [substeps] Visit a doctor if you suspect they may be a head bug.', '[substeps] Any mosquito bites may also indicate bedbug bites. [title] Look for insect bites that bore holes into the skin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}