{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c4f961ff-73b2-5569-af07-514958afa73e","task_key":"default--validation--45027","task_revision_id":"1","upstream_id":"45027","short_description":"HellaSwag validation 45027","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to make a smart car surveillance system using a mobile phone [title] Choosing mobile phone for the system get a gsm mobile phone which has auto-answer and mute functions. [step] It should have good reception and a perfect microphone. The ones with small external antennae are the most suitable.\",\"endings\":\"['[title] Noting the imei number take down the imei number of your mobile phone. [step] This number is usually printed inside the battery compartment, located beneath the battery of the mobile phone.', 'The vertical antenna should be 1.5 inch (3.9 mm), the video antenna should be 1.5 inches (3.9 mm). [title] Bring in your basic supplies and start planning.', '[title] Have your friends send you reports about surveillance. [step] Always tell them what is going on with the different types of devices, e.g.', '( so are bigger phones ) [title] Gather the names of relevant individuals. [step] Each person should have a unique and distinct title.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}