{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f5a1ed6-e9fa-5620-a2cf-64bfd960995b","task_key":"default--validation--26452","task_revision_id":"1","upstream_id":"26452","short_description":"HellaSwag validation 26452","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to tell if your spouse is lying [title] Look for excessive blinking. [step] This may occur when you start talking to him about an uncomfortable topic. If you are talking to your spouse about something he is lying about, he may start to panic.\",\"endings\":\"['Blinking, though, is a sign of lack of emotion and excitement. [substeps] A slow and rapid blinking of the eyes often means that your spouse is extremely nervous.', 'Pay attention to any flashing of blue or purple eyes when talking to him. [substeps] You might also start blinking if your spouse is lying about something.', 'Occasionally the lying spouse may actually slow down his blink rate going into the lie and speed it up afterwards. [substeps] Blinking frequently can be a reflex in this case.', \\\"However, if you are making an effort to win his attention, such as asking about his day, you should notice that your spouse is constantly blinking like a dog; this can be a sign of an anxiety disorder. [substeps] Try to observe your spouse's eye movements so that you can tell if something is bothering him.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}