{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e4861a8e-9519-5987-a9ba-8c9821d5438e","task_key":"default--validation--34261","task_revision_id":"1","upstream_id":"34261","short_description":"HellaSwag validation 34261","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to travel by train from los angeles to san francisco [title] Plan a leisurely trip. [step] The coast starlight it not prized for its on-time record, but it is famous for its beautiful views. The trip itself takes approximately 12 hours, from 10 a.m. to 10 p.m., but that can vary based on amtrak train schedules.\",\"endings\":\"['On-line train trains run from about 6 p.m. to 3 p.m., so expect a sightseeing option.', '[title] Book your ticket on the amtrak website. [step] You are encouraged to book early and during the week.', 'For rail travel : [substeps] Plan to travel at the end of the day. Areas where sunset express travel runs, know to take a train from inside los angeles to san francisco station.', 'California rail systems offer a variance of between 12 a.m. and 11 p.m., available for all major rail travel companies.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}