{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"61cd4784-b91d-500e-acfe-61144cfd2d3b","task_key":"default--validation--46533","task_revision_id":"1","upstream_id":"46533","short_description":"HellaSwag validation 46533","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to make a multiplication tables activity board using plastic water bottle caps [title] Plug in hot glue gun, cautiously. [step] Allow the hot glue gun to warm up. [substeps] Don't let your kids use the glue gun themselves.\",\"endings\":\"['[title] Remove cover cap in step one, as this can quickly melt the cover casing. [step] Clean cap and screw it back into place.', 'As they get more proficient with the machine with the wires stuck in it, they may well realise that another method is not so helpful. The only way to avoid this is to select a theme board if you do not have one.', 'Usually, they jump around a bit and bleed, and they think they can solve the problem. If your child is very young, get a few old books and wear them while working on the board.', \\\"It's hot, and can cause burns. [title] Use a glue stick to add glue to the construction paper.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}