{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"80b0a5e8-d3a9-5bf2-bb7b-79907bf739bf","task_key":"default--validation--42672","task_revision_id":"1","upstream_id":"42672","short_description":"HellaSwag validation 42672","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make coffee with a coffee press [title] Measure the coffee beans. [step] To ensure that you wind up with the right amount of grounds for your coffee, you must carefully measure out the coffee beans. For 32 ounces (946 ml) of coffee, which is approximately 3 to 4 servings, use a measuring cup to measure a ½ cup (51 g) of beans.\",\"endings\":\"['[substeps] The standard size for most coffee presses is 32 ounces (946 ml), but you can make fewer servings if you prefer. For a single serving, measure 2 tablespoons (13 g) of beans.', 'For 5 ounces (165 ml) of black coffee, that also includes 5 servings. [substeps] Double-check that the coffee beans you use are ok to drink in the coffee press.', 'To measure two or four servings, use a measuring cup with a button. You should use only a pinch of sugar if your coffee contains sweet ingredients, and a pinch of salt if your coffee contains milk.', \\\"[substeps] If you don't have a measuring cup, measure out 32 oz (119 ml) of coffee beans and put them into a measuring cup. Transfer that measurement to a scale or a disposable measuring cup.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}