{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ee3db27b-4ba3-5124-a872-26fcd27bc432","task_key":"default--validation--23117","task_revision_id":"1","upstream_id":"23117","short_description":"HellaSwag validation 23117","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use a tap [title] Determine the size and depth of the thread that you want. [step] Will go into this hole? [title] Refer to a technical publication for the exact measurements for hole size. [step] For example, if you want to use a 1/4-20 screw then you must use a 7 drill bit.\",\"endings\":\"['There are some larger drill bits, but use whatever 10/20 nuts or markers you have available, with the $10 electronic spirit plug provided. [title] Carefully examine the hole in the center of the screw (the one also known as the \\\" hold \\\") and identify the depth measurement.', 'Your set of taps may also state the appropriate hole size to use with each tap. [title] Drill the hole to the correct diameter and depth.', '[title] Invest in regular, eye-catching thread that is striped or embellished. [step] This will provide more stability during your tap.', '[title] Measure the fingerboard to set the depth. [step] Use a ruler to make sure that you have a square spot where you can easily fit a drill bit into.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}