{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e75cc2a-fe7a-5d28-b035-569e3eb189d6","task_key":"default--validation--29138","task_revision_id":"1","upstream_id":"29138","short_description":"HellaSwag validation 29138","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to work a room [title] Enter the room with a clear goal. [step] You should know why you have decided to work the room. Your networking goal may be quite focused, such as trying to get an invite to a particular meeting, or you may simply hope to meet three new people with shared interests.\",\"endings\":\"['Whatever the reason for your designated task, make sure you have something to occupy your time completely so you can head to your goal you have put thought into. [substeps] Look at a glass of water by the door.', \\\"Learning about each other's interests will greatly help you become more comfortable. [title] Describe the job topic in paragraph form.\\\", 'Whatever your goal, you should keep it in mind as you enter the room. [substeps] If you are hoping to meet particular people at the event, you should do some research on their work beforehand.', 'Whatever the reason you are setting yourself up, you will need to push through the crowd and approach each particular candidate individually. [substeps] Experience with the company, the great culture, the developmental advantages of each particular stat, and the unique skills you possess.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}