{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b4727c8a-80fa-55d5-9f6b-84b81ca39cae","task_key":"default--validation--42967","task_revision_id":"1","upstream_id":"42967","short_description":"HellaSwag validation 42967","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to be honest with yourself [title] Get into the right mindset. [step] Be open to doing a self-assessment, which could be a useful problem-solving tool. You should approach it without shame or blame.\",\"endings\":\"['If you tell yourself, \\\" this is so wrong that i embarrassed myself, \\\" that\\\\'s not true. You need to set goals and be motivated to achieve them.', \\\"You don't have to be brutally honest with yourself. Instead, be gentle and kind while still being honest.\\\", \\\"Use your good thoughts on the positive, and your positive thoughts on the negative. This doesn't mean you have to wait for the ordeal to end.\\\", \\\"If you aren't one for being honest, you may be confusing yourself. In your head, it should feel as though you are sitting in a trap with little room for improvement.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}