{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"humaneval","formal_name":"HumanEval","introduction":"HumanEval asks for a Python function body from a docstring and signature, checked by hidden unit tests. It is the original functional-correctness benchmark for code generation.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/openai/openai_humaneval","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d706bc55-adc7-5926-8fa3-acdfcda5da1c","task_key":"openai~5fhumaneval--test--d706bc55-adc7-5926-8fa3-acdfcda5da1c","task_revision_id":"4","upstream_id":"","short_description":"def compare(game,guess):","config":"openai_humaneval","split":"test","body":"{\"entry_point\":\"compare\",\"prompt\":\"\\ndef compare(game,guess):\\n    \\\"\\\"\\\"I think we all remember that feeling when the result of some long-awaited\\n    event is finally known. The feelings and thoughts you have at that moment are\\n    definitely worth noting down and comparing.\\n    Your task is to determine if a person correctly guessed the results of a number of matches.\\n    You are given two arrays of scores and guesses of equal length, where each index shows a match. \\n    Return an array of the same length denoting how far off each guess was. If they have guessed correctly,\\n    the value is 0, and if not, the value is the absolute difference between the guess and the score.\\n    \\n    \\n    example:\\n\\n    compare([1,2,3,4,5,1],[1,2,3,4,2,-2]) -> [0,0,0,0,3,3]\\n    compare([0,5,0,0,0,4],[4,1,1,0,0,-2]) -> [4,4,1,0,0,6]\\n    \\\"\\\"\\\"\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/openai/openai_humaneval","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}