{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e65f0e99-fd77-5696-b68b-b3fa0b18a1c2","task_key":"default--validation--41748","task_revision_id":"1","upstream_id":"41748","short_description":"HellaSwag validation 41748","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to answer what interests you about this job [title] Do your research. [step] Before an interview, you should always take time to research the company. Find out everything you can from the company's website.\",\"endings\":\"['Look for anything that seemed disappointing or faulty to you when you interviewed. [substeps] Avoid asking general questions like, \\\" what did the company like about this place look like? \\\" rather, go deeper into what it looked like as you began your job.', 'Often, interview people already in the company that have applied to that position. You might be able to learn a lot from a few online sources and on-the-job interviews.', \\\"Learn about the company, what types of jobs they have but also the job placement area. These information can help you find an interview if you haven't done enough research in the past two years.\\\", \\\"You can also do an internet news search to see what the company has been up to lately. Don't forget to look at things like the company's mission statement to get an idea about what is valued there.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}