{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5aa9268f-e176-5df4-8d1e-a29b26c2372e","task_key":"default--validation--2747","task_revision_id":"1","upstream_id":"2747","short_description":"HellaSwag validation 2747","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to work out salary increase percentage [title] Subtract your old salary from your new salary. [step] Say you made $45 , 000 per year at your old job and that you accepted a new position making $50 , 000 per year. This means you would take $50 , 000 and subtract $45 , 000.\",\"endings\":\"[\\\"$ 50 , 000-$45 , 000 = $5 , 000. [substeps] If you receive hourly pay and don't know your total yearly earnings, you can simply use the old and new hourly rate in place of the salary.\\\", 'To get your current current salary, you must multiply $50 , 000 by. 50 which equals $5 , 000 you worked out together.', 'You would subtract $50 , 000 from the $45 , 000 to get your new salary. [substeps] For example, if your current salary is $150 , 000 you would subtract $150 to get your new salary.', 'The remaining $70 , 000 would equal $100 , 000 in your new salary. [substeps] If you have less in your previous salary than you did in your current position, you should subtract $100 , 000 from your current salary and make the difference.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}