{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b164bdf0-b4b4-5eba-8918-c5f9a760b588","task_key":"default--validation--38654","task_revision_id":"1","upstream_id":"38654","short_description":"HellaSwag validation 38654","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress with sensation with a sari [title] Wear the sari first. [step] Buy a ready made sari, with the pleats at the skirt done. Also, the, \\\" pallu \\\", or the part which goes above the breast should be silken and decorated so that extra pleats won't be needed.\",\"endings\":\"['[title] Wear a top or high dress with pleats. [step] Once you are comfortable with the puffed up look, start wearing your sari.', 'It can be a dark, realistic style or a more free-spirited one. Black and/or brown permanent stencil the sari, and add the other colors like pink.', \\\"The color should now be in the ladies' colours. Show off your sari while bringing people in your party an indication that you are happy.\\\", '[title] Start out by wrapping the part with the pleats around your waist. [step] When the ends meet, the skirt waist should have two strings.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}