{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"32c1bf4e-154b-50d7-8ba1-220afbd9c047","task_key":"default--validation--16819","task_revision_id":"1","upstream_id":"16819","short_description":"HellaSwag validation 16819","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean a diamond ring on a low budget [title] Find an old pot (the smaller the better). [step] It is of clay, that's better. Yet it does not matter if it is of iron either.\",\"endings\":\"['[title] Get your bit of duct tape, also go pipe this stuff in the pot for quick welds. [title] Watch out for a lot of smells (like marijuana).', 'Use a clean kitchen sponge to scrub the ring loose. Rinse out the sponge after each use and make sure to dry completely.', \\\"It's still important to extract the silver stopper during the cleaning process. [title] Pour a square of activated non-iodized baking soda into the fork, making sure it is evenly distributed on the ring.\\\", '[title] Fill the pot with dishwashing soap. [step] ( enough to submerge the ring).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}