{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f0841c07-d429-54d1-b434-b9b8ffd62116","task_key":"default--validation--3039","task_revision_id":"1","upstream_id":"3039","short_description":"HellaSwag validation 3039","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to give a sponge bath [title] Fill two basins or washtubs with warm water. [step] One is used for washing, and the other for rinsing. The water temperature should be 115 degrees f (46 degrees c) or less.\",\"endings\":\"[\\\"You want it to be comfortable to the touch, but not too hot. [title] Choose soap that's easy to rinse away.\\\", 'Hot water will be less effective at scrubbing the soap out of the sponge, so lukewarm water will be most effective. Try water that is both warm and hot and is not lukewarm.', \\\"Be sure to use a setting on your tap for the temperature of your soap to be lower than average while you're bathing or sponging. Fill the bowls a little more than halfway with warm water, though keep the edge away from your skin.\\\", \\\"[substeps] If you're using simple water, use warm (but not scalding) water only, not warm (but don't use ice water). Sponge baths are considered safer when done by a professional.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}