{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e4cc7e7-fb33-5d58-9e08-ba7c89112577","task_key":"default--validation--39714","task_revision_id":"1","upstream_id":"39714","short_description":"HellaSwag validation 39714","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to remove dead skin from feet [title] Mash a banana and rub it on your feet. [step] Make sure you use bananas that are as ripe as possible, almost to the point of being too ripe to eat. Place one or two bananas in a bowl.\",\"endings\":\"['Mash them well so that they are mixed with the banana. [title] Perform a foot depilatory.', 'Use a fork or masher to mash then into a smooth paste. Apply to feet and leave it for about 20 minutes.', 'Then, mix in three tablespoons (90 ml) of honey. Place one banana in each bowl and mix in two tablespoons of baking soda, one tablespoon (15 ml) of honey and one teaspoon (1 g) of baking soda.', '[title] Mix an apple juice with your foot. [step] Juice contains both antioxidants and ph, so splashing it on helps remove dead skin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}