{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a8045dd4-6142-5183-9eec-d24be8a5389b","task_key":"default--validation--43104","task_revision_id":"1","upstream_id":"43104","short_description":"HellaSwag validation 43104","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use a magimix [title] Place the bowl on the motor housing with the handle to the right hand side. [step] The keyholes in the bottom rim should locate over the lugs on the motor housing (for cuisine system only). [title] Turn the bowl counter-clockwise as far as it will go, making sure it firmly locks in position.\",\"endings\":\"['[step] If on a bike the turns will be all the way clockwise, if not, keep turning until the wheel starts moving. Bring your left wheel right until the wheel starts moving away from the motor housing.', '[title] Turn the key clockwise as far as it will go, and turn it again, trying to slide it in and out of the housing. [step] Hear or feel your cpm rising with each revolution.', '[title] Slide the mini-bowl down onto the motor spindle. [title] Place the mini-bowl blade onto the motor spindle.', '[title] Press the button (for approval). [title] Ensure the cassette is turned as far clockwise as it will go, then turn it right hand clockwise as far right as it will go (for appliance use only).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}