{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5874e4e0-af04-5054-8a4c-e1bfb23f323f","task_key":"default--validation--38894","task_revision_id":"1","upstream_id":"38894","short_description":"HellaSwag validation 38894","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to cut ceiling tiles [title] Purchase the ceiling tiles if necessary. [step] If you are conducting a replacement job on an existing ceiling, you will want to match the ceiling tile style to the existing tiles. The easiest way to do this is to order much more than you need during the initial installation and store them for later use.\",\"endings\":\"['[title] Remove the ceiling tiles on the top 2-4 trim pieces to expose the paint underneath. [step] The paint will thin and give the tiles a polished look.', 'If you are buying new tiles, start by removing them from their package and letting them rest inside overnight. The tiles need time to adjust to temperature and humidity conditions, as they can expand or contract a bit.', '[substeps] To finish the job: place the ceiling tiles one strand at a time in the same way you did the surrounding tiles. Grab each tile, then use a hammer or box to pound nails into the tile.', '[title] Cut the ceiling tiles to the correct size and hold in place with small snips or a drill. [step] This will allow the tiles to stay on the ceiling and makes it less likely that mites or small insects could escape.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}