{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"44aafa7f-9b88-5f55-9054-64dfbb5ef237","task_key":"default--validation--32147","task_revision_id":"1","upstream_id":"32147","short_description":"HellaSwag validation 32147","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wind a watch [title] Lay the wristwatch on a table. [step] Remove the wristwatch from your arm or storage. If you wind a watch while it is on the arm, with the watch tight against the skin, the winding motion may not be effective.\",\"endings\":\"['[substeps] Winding a watch while attached to your arm places strain on the mechanics in the watch because of the angle on your arm and the stem. Locate the stem, which will be pulled out to wind the watch.', 'Try to wind the watch instead of your wrist: when you unscrew the watch, the blood will separate from the loosened hair at the bottom of the watch. In a particularly humid or rainy area, turn the watch part downwards at eye level.', 'If you unscrew the watch, the winding movement is more likely to cause problems. [substeps] This method will only work if the watch is small enough to fit on your wrist.', \\\"A loose watch will often take more effort to wind than a watch with a relatively flexible wrist. [substeps] If the watch isn't sufficiently loose to fit on your wrist, then try to loosen and unscrew it.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}