{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"994f8567-b49f-505a-8d57-4f287c06edb8","task_key":"default--validation--6667","task_revision_id":"1","upstream_id":"6667","short_description":"HellaSwag validation 6667","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to figure out your yearly salary [title] Find out your hourly wage. [step] You probably already know how much you make per hour. For example, you might make $15/hour.\",\"endings\":\"[\\\"You can't make half those hours. However, one way to figure out what hourly wage you make is to check with your job pay sheet.\\\", \\\"But if you don't, you'll need to find this information. [substeps] Your hourly wage should be listed on your pay stub, if your employer provided one.\\\", 'This figure will probably change based on what type of work you do and what position you hope to be in. For example, if you have a babysitting job, you can determine how much you make per hour based on what role you play in the life of the baby.', 'Write down your hourly wage first, then the daily or monthly average you earned. [substeps] For example, you might have published your hourly rate first, so if you have a flexible accounting philosophy, you would have been evaluating your hourly wage.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}