{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"58460b8c-3cfa-5617-bad6-ee8e0ee4013b","task_key":"default--validation--33915","task_revision_id":"1","upstream_id":"33915","short_description":"HellaSwag validation 33915","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to repair hail damage [title] Park your vehicle in a sunny area. [step] This is a popular technique with car enthusiasts. Heat will cause the metal to expand.\",\"endings\":\"[\\\"As the metal expands, the small dents will pop out. Using nature's oven will take a long time if you aren't in a very warm place.\\\", 'Consider laying on some metal, so that you can run over it to repair the damage. Not only will this also provide immediate relief from any strain on your tires, but it is also a safer way to repair them.', 'Thus, make sure your vehicle is positioned at the right heat while you park. [substeps] Ensure that your vehicle is equipped with a well-insulated interior.', 'Pushing the temperature up by about 40 ° f (15. 2 ° c) will cause the metal to contract.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}