{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"22884c89-5b76-592f-abbe-47396b277761","task_key":"default--validation--43303","task_revision_id":"1","upstream_id":"43303","short_description":"HellaSwag validation 43303","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to fix nail clippers [title] Insert the pin. [step] A pin or rod-like piece holds the handle of your nail clippers in place and allows you to clip your nails. Locate the pin and then locate the hole near the clipper blades.\",\"endings\":\"['Take the pin and insert it through the hole. Make sure that the notch in the pin is on the upper side of the clippers.', 'At first, you may have two holes on either side of the pin. Find the hole closest to the blade by gently bending the rod back and forth.', 'Place the pin inside the hole and the rod into the hole. [title] Make sure the edge of the pin is flush with the bottom edge of the nail.', 'The hole should be facing the top edge of the teeth. Angle the pin until it is in the middle of the blade.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}