{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9a8dcd29-d600-5fbc-9379-14f9b87b019f","task_key":"default--validation--15123","task_revision_id":"1","upstream_id":"15123","short_description":"HellaSwag validation 15123","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to diversify sandwich fillings [title] Make cheese sandwiches more interesting. [step] While cheese sandwiches are an easy mainstay for lunchboxes, they can be made a lot more interesting than just simple cheese. Here are some ideas : [substeps] Change the cheese.\",\"endings\":\"[\\\"If you've been eating only one kind of cheese, branch out and try other cheese varieties, especially those with stronger flavors, and cheese types you've never tried before. Red pepper, cress, or radish to mild cheese for a piquant taste.\\\", 'Go for cheddar, cognac, or mojito. Put a corner of cheese on your sandwich for variation.', 'The types of cheeses are the best for sandwiches. Cheddar, rumansey, and gruyere are both great to use because they both fit into your sandwiches.', \\\"For example, change the red cheese for one sandwich in bold and replace the white cheese with cheddar instead. That is, you could eat everything in a sandwich you've made.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}