{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1fe40336-01c8-5bcb-bd2d-250fe7b5e448","task_key":"default--validation--27790","task_revision_id":"1","upstream_id":"27790","short_description":"HellaSwag validation 27790","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to present the outside of your house for a quick house sale [title] Make sure your front door looks its best! [step] A front door should be an impressive and strong object. It should also be in mint condition. After all, buyers will come nose to nose with it.\",\"endings\":\"[\\\"[title] Make sure you let your front door know you're excited about the sale. [step] That way, if a buyer comes up to your home and looks at it like it's a beautiful piece of art, they'll be more likely to buy.\\\", 'Jobs to consider if needed : [substeps] Repaint in a conservative colour (black, blue, white, red or green). Polish all knobs and knockers or replace if dated.', 'Using the door allows buyers and sellers to see each other as they walk through the door. It also seals the rooms inside from outside.', \\\"It will be shocked if they can't wrap their hands around it. [title] If you have an extensive yard that gets sandy and will not stand easily, consider making a motion stop sign.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}