{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e161f532-1a27-5d2a-9e40-7da2836b5081","task_key":"default--validation--42263","task_revision_id":"1","upstream_id":"42263","short_description":"HellaSwag validation 42263","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose a leather jacket [title] Compare the most popular jacket styles. [step] Bomber jackets, double riders (also called \\\" schott perfecto \\\"), and moto \\\" racer \\\" jackets are the most common, classic leather jacket styles. The best jacket style for you will be the one that matches your taste.\",\"endings\":\"[\\\"[title] Contact the seller who produces the jacket. [step] Petually be aware of the ups and downs of the typical leather jacket designs, so that you don't miss anything.\\\", '[substeps] Many european jackets have dual buckles, which will make them easier to open or close. Bomber jackets have adjustable straps with an adjustable clasp to hold the jacket in place without pressing on the zipper or resulting tear.', '[substeps] If you love racing, take an image of your racing style for inspiration. You can even try something like the classic bomber jackets with star stars.', 'Start by looking at variations of these three styles. [substeps] The double rider is considered a classic american leather jacket.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}