{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b82b420e-f4aa-587d-a601-9f24dd4305d7","task_key":"default--validation--38297","task_revision_id":"1","upstream_id":"38297","short_description":"HellaSwag validation 38297","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to fill raised garden beds [title] Calculate how much soil you will need. [step] Measure the dimensions of your garden bed using a tape measure. You will need the length, width, and depth of the bed.\",\"endings\":\"['You can then divide the numbers between different beds by an estimate. The resulting article will provide an estimate of the length, width, and depth of your raised garden bed.', 'Multiply the length, width, and depth measurements by the following formula : [substeps] Length: you need 24 inches (61 cm) of space for a raised bed. Depth: you need 12 inches (61 cm) of soil for a raised bed.', 'Place those measurements into an online soil volume calculator. Find one of these by searching on the internet.', '[substeps] Above the soil line is the area to place the tiller underneath the soil line. Multiply the length, width, and depth of the bed according to your soil measurements.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}