{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d53a6f8-2973-57f0-93f8-19b9c066cddc","task_key":"default--validation--41761","task_revision_id":"1","upstream_id":"41761","short_description":"HellaSwag validation 41761","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to strengthen character [title] Be more honest. [step] Honesty is a key component of your character. Show others that you are honest by making your actions match your words.\",\"endings\":\"[\\\"[substeps] If you tell a secret, don't just admit it so to people that you can still stay present. Before sharing this information, make sure that your words reflect who you are and who you are as a person.\\\", 'Also, tell the truth without being dishonest. Cheating shows people that you have a problem.', \\\"For example, if you tell your partner that you will be more supportive of their career, show them you mean it. You could make a point to ask them about a big project they're on or you could offer to take over dinner duty during a particularly busy time.\\\", 'If you are dishonest or uncomfortable telling people, you may find yourself in deeper trouble. If you are not honest with others, you may find yourself in deeper trouble.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}