{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0be9c261-b28f-5730-a11b-993e07b5980a","task_key":"default--validation--11565","task_revision_id":"1","upstream_id":"11565","short_description":"HellaSwag validation 11565","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to stain cabinets [title] Remove all of the contents from your cabinets. [step] You will need them to be stored in another place for about a week. [title] Remove the doors and hardware.\",\"endings\":\"[\\\"[step] You will want to remove the cabinets first, so you can have the entire surface cleaned. You can leave the screws in, so the grime won't get into the area you want to stain.\\\", '[step] These and any hardware in these cabinets also need to be cleaned and reinstalled so you can go caulk-dried or microchipped. Acrylic paint can be purchased at major retailers and will work to seal the surface.', '[step] Using a small tool, gently pull apart the cabinet doors to reveal the interior inside of each corner. [substeps] Remove any knobs on the sides or switches to check under or below them.', \\\"[step] If you don't take all the hinges and knobs off, the stain won't penetrate the entire area, and you may have inconsistencies and bubbles in your finish. Keep all the knobs and hinges in sealable plastic bags with the screws that attach them.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}