{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"93989b69-a3bd-5f93-b409-749124fc4fdb","task_key":"default--validation--13466","task_revision_id":"1","upstream_id":"13466","short_description":"HellaSwag validation 13466","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to set up a pa system [title] Be sure to be using suitable equipment. [step] If renting the pa system, take a look at the components to make sure they are in decent condition. While you may not be able to fully test the system quickly, you will be able to notice broken speaker stands and things that can be a concern.\",\"endings\":\"['[title] Be sure to set the speakers in the right location. [step] The speakers should be in front of, and never behind, the microphones, to avoid feedback.', 'Make sure that there are no obstructions that will block the system from accessing the infrastructure. [substeps] If a vehicle is blocking the system, it is a good idea to work with a supervisor or get someone to help you out.', 'If a local hardware store does not sell pa system systems, try a hardware store. If there is a good mix of equipment and hardware there will generally be a good balance of electronic and non-electronic components.', \\\"[substeps] In a conflict-resolution dispute, it's advisable to use equipment for the accommodation functions of the pa system. In a negotiation situation, try to resolve injustices before the confrontation.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}