{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f5af13c-fdb8-50bc-b59e-ebc27bce0bdf","task_key":"default--validation--26392","task_revision_id":"1","upstream_id":"26392","short_description":"HellaSwag validation 26392","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to shoot a handgun [title] Learn the difference between a revolver and a semi-automatic pistol. [step] These are the two basic types of handguns. The handguns you would see in a cowboy movie are revolvers.\",\"endings\":\"['They must be round 25 caliber (with the cylinder on the first side and the cylinder in the middle). They contain the primed outer spring, which is unlocked when the gun is the correct barrel length, with a switch located on the inside of the cylinder that places the switch over the front side of the cylinder.', 'The cartridges in a revolver are loaded into a cylinder in the middle of the frame of the gun. A capacity of six is most common but some hold five and are designed to fit smaller hands.', 'As the phrase implies, you have to pull up your weapon for a shot to test the appropriate trigger mechanism. Find the lever at the side of the gun.', 'The semi-automatic is the most common type of gun with a recoil. The semi-automatic is the most common type of handgun you would see in a revolver.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}