{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6f8dba0b-eb68-5081-a958-b65534146f2c","task_key":"default--validation--17074","task_revision_id":"1","upstream_id":"17074","short_description":"HellaSwag validation 17074","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a beach bag using a shower curtain [title] Make a pattern for your beach bag out of newspaper. [step] If you want the bottom of the beach bag to be rounded, fold the newspaper in half before cutting so that once the pattern is unfolded. That way, it will be full size, with both bottom corners cut in exactly the same shape.\",\"endings\":\"['Make sure that you cut the pattern piece lengthwise, or else the material may look lumpy, lopsided, or messy. [title] Fill your plastic or construction bag with sand.', \\\"[substeps] Patterns for beach bags can be found online or in your local library. Given the size of the shower curtain, you can make your beach bag as large as you'd like.\\\", \\\"This will ensure that the bag holds better through the day when you're trying to find the right place to keep your dinner leftovers. [title] Take your sheet with you when you make your beach bag.\\\", 'Use a marker to draw a well in the middle, then cut the pattern to size. You can find patterns for shower curtains online or at a photo or craft store.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}