{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bcd0b49c-cd8c-5d6c-9c7d-fca135fc8a03","task_key":"default--validation--16621--variant-ec89377610e9","task_revision_id":"1","upstream_id":"16621--variant-ec89377610e9","short_description":"HellaSwag validation 16621","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to buy an android tablet [title] Decide how you will use the tablet. [step] This question defines the remaining steps and is the most important. [substeps] Is the tablet only needed for watching videos, instagram, and facebook? If so, most tablets today can handle that.\",\"endings\":\"[\\\"Is the device capable of downloading videos? If so, some on the device may require special drivers. Apps like gmail (for mobile phone users) and browsers like android tablets are likely won't work on android tablets with tablet fobs, as they don't guarantee you everything you need.\\\", 'Android tablets with an offline view are much better. Though android tablets are pretty low-tech, they can easily handle real-time broadcasts.', 'Users can save money by purchasing lower-end tablets for these purposes. Is the tablet going to be used for multimedia and gaming? Then a higher end tablet will be in order.', 'Note, however, that since android tablets are inexpensive, you will get paid for every $600 you spend. Doing this will open up a variety of stores.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}