{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bb0a3ff4-30bf-539a-a9c6-6689fc75e289","task_key":"default--validation--30999","task_revision_id":"1","upstream_id":"30999","short_description":"HellaSwag validation 30999","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to choose a university (uk ) [title] Make sure that you start early enough. [step] Ideally you should start looking for a university about 18 months before commencement of the course and start applying about a year before commencement. British universities operate on first come, first served basis (more or less) and you do not want to be left out of your favourite course because you did not start looking early enough.\",\"endings\":\"[\\\"[title] Interview for the university's guidance council (cph). [step] When you're learning about a university, there are not as many guidelines.\\\", '[title] Decide if you want to do a distance learning course (such as open university). [step] These are becoming increasingly popular as they require much less commitment.', '[title] Get the necessary registration from the university. [step] In some places, it is not allowed to register for levels of a course without applying and might sound odd.', 'Other requirements are also set by the university, so make sure they are like that. Letters of recommendation and scholarships are very important.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}