{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e5128550-981f-5186-9076-85f764e6e8ff","task_key":"default--validation--14538","task_revision_id":"1","upstream_id":"14538","short_description":"HellaSwag validation 14538","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to breed a horse [title] Calculate the financial cost of impregnating your horse. [step] Know that there is a considerable amount of cost involved with breeding a horse. There are fees for the stallion to breed with the mare, plus insurance costs to protect you should the mare kick out and damage the stallion, plus transport costs to get the mare to the stallion and back.\",\"endings\":\"['In taking all this into account, in comparing horse breeding costs, it is wise to calculate any additional costs that you may incur with a mare. [substeps] To bring costs up, you can divide this amount by the estimated cost of breeding.', 'Make sure you factor in full costs of labor in your calculations. You might also want to consider how much the mare may pay during labor and if you will need to provide insurance to ensure that labor is plentiful.', 'Lamb run is a surprisingly affordable distance for large companies (generally horses looking for higher prices). Breeding a mare in western europe can cost anywhere from at least $2 , 000 and $20 , 000.', 'Bear in mind that she may not become pregnant on the first attempt and so this may need to be repeated. [substeps] Instead of using a stallion, you can also opt for artificial insemination.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}