{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b9b4edda-c624-5624-aebe-5c8204309c5a","task_key":"default--validation--23764","task_revision_id":"1","upstream_id":"23764","short_description":"HellaSwag validation 23764","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to attract rats [title] Leave some food sitting out. [step] Rats aren't picky eaters-you can use just about any type of edible item as rodent bait, including meats, cheeses, fish, moldy bread, and produce that's past its prime. Sweet offerings like fruit are among their favorites, particularly berries.\",\"endings\":\"['[substeps] Rats prefer foods with strong odors, which they use to sniff out a meal. For this reason, spoiled items may be more effective for drawing them out than fresh ones.', '[substeps] Rats can also be attracted to dried mice and other rodent-friendly things in the recipes of experts. [title] Offer onions, celery, and garlic.', '[title] Try to attract rats to small spaces. [step] Rats are naturally very territorial creatures-they can go chasing after every living thing in their home.', \\\"[substeps] Don't leave small pieces of fruit in your rats' homes, as they like to eat when they're hungry. Rats do best when they have plenty of fresh food available.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}