{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b7ef5b41-4f32-5170-a15a-cd03c2ad2f5d","task_key":"default--validation--27031","task_revision_id":"1","upstream_id":"27031","short_description":"HellaSwag validation 27031","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a lotion bar with almond oil [title] Visit the craft or health food store to compile ingredients. [step] You'll need a 1 lb. Bar of beeswax, ½ cup of almond oil and ½ cup of coconut oil.\",\"endings\":\"['[title] Locate your double boiler or create your own using two pans and a little water. [step] This will be the tool you use to melt the bars down to create your own slice of skin paradise.', 'Make sure you stock up on the ingredients that you will use to make your bar of coconut cream. [substeps] You will want a blender, bugle bowl, or electric mixer to process and put the ingredients in, in order.', 'Apply the mixture to your skin, paying attention to any blemishes or discolorations. [title] Scrape off any excess honey or milk.', 'You can also find a good dairy-free menthogenic lotion at the store. [title] Break the sunglass shell and shave it off.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}