{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bab8ca2e-d347-52c8-badc-d12819072779","task_key":"default--validation--17260","task_revision_id":"1","upstream_id":"17260","short_description":"HellaSwag validation 17260","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to keep air out of your baby's bottle [title] Prepare the formula or milk in advance. [step] Mix the milk or formula in a separate bowl or cup. When you prepare the formula in the bottle, there is an increased chance that air bubbles will form.\",\"endings\":\"['[substeps] Make sure to stir the formula slowly so that you minimize the number of air bubbles that form. [title] Avoid pouring the milk from high above the bottle.', 'For months you will need to leave the bottle warm to prevent air from getting in the bottle and ruining the quality of the formula. [substeps] If you do not already prepare the formula in advance, contact the manufacturer about switching to a brand new formula if you are not sure what brand to buy.', 'Make sure you are not stir in all of the formula at once. [title] Insert one inserted bulb into a reservoir.', 'Setting up the milk well ahead of time will help prevent that. [title] Keep in mind that milk always needs to be warmed to the right temperature.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}