{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"45779a31-28b7-536d-80a2-e0268ab7bb27","task_key":"default--validation--41755","task_revision_id":"1","upstream_id":"41755","short_description":"HellaSwag validation 41755","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to disable a seat belt alarm [title] Choose a seat belt alarm stopper. [step] There are two basic products on the market that are referred to as a seat belt alarm stopper. One is simply an unattached seat belt clip that can be inserted into the receptacle.\",\"endings\":\"['The other is a latch that sticks open. [substeps] The latch is used to secure the buckle to the belt only, which prevents it from taking up space along the top of the belt.', 'The other is an extender that clips into the receptacle, but also provides an additional receptacle so that you can use your seat belt without removing the alarm stopper. [title] Purchase your seat belt alarm stopper of choice.', \\\"The other is an extension on the side of the seat that you can attach to the door frame as a safe option. The capacity of the seat belt when inserted is determined by the wearer's head height.\\\", 'The other is a lock bolt that goes around the foot to prevent the belt from tightening. To ensure you have one built into your vehicle, either choose a special model or select a specially designed seat belt stopper.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}