{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6862f5b3-3f68-5ca5-879b-049b256f3f39","task_key":"default--validation--28249","task_revision_id":"1","upstream_id":"28249","short_description":"HellaSwag validation 28249","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to identify pearls in vintage jewelry [title] Look for rings around the drill hole. [step] A fake pearl will not have any rings or or ridges around a drill hole. Put the pearls under a light and use a magnifying glass to inspect the drill hole.\",\"endings\":\"['If the holes are smooth, then the pearls are likely fake. [title] Examine for variations in size and color.', 'Be careful not to squeeze your fingers too much or else you may break the actual pearl. [substeps] Your fake pearl will have rings, ridges, engraving under it.', 'You may also see splashes of flesh. Pearls on a drilling ground are usually collected from spills, not from hitting the drill with the jewelry hammer.', 'You can also check for tiny pearls by scanning the graduated tip of the drill. This should reveal a tiny pearl attached to a thread.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}