{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a5f83e84-9bf8-579f-9b4c-58ffe5d60b8c","task_key":"default--validation--1162","task_revision_id":"1","upstream_id":"1162","short_description":"HellaSwag validation 1162","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a clothesline to use without hangers [title] Find the smallest, thinnest surgical tubing you can find. [step] If not surgical tubing, you can use some form of long rubber band. If you can't find something like these, buy a box of the heavy gauge rubber bands that the post office uses around bundles of mail.\",\"endings\":\"[\\\"You will want about 20 feet (6.1 m) of the item in it's unstretched condition. You will need two small gate hooks from any hardware store, too.\\\", 'This should give you enough thread to wrap the looped metal band around the center. [title] Find a sturdy post or some other solid object to place the wire around.', 'It can cost anywhere from $3000-$300. [title] Pull out the cap of the tube and gently untangle it so that the coat line is free from the hose.', '[title] Visit your local post office to obtain scissors and wire cutters. [step] You can use just about any type of scissors to cut your pole.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}