{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bigcodebench","formal_name":"BigCodeBench","introduction":"BigCodeBench poses 1,140 function-level tasks drawn across 139 libraries. It tests whether a model can compose several real APIs correctly rather than write one self-contained function.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://bigcode-bench.github.io/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"f12070cd-8001-50c3-9511-0e5dd17ccfa3","task_key":"default--v0~2e1~2e0~5fhf--f12070cd-8001-50c3-9511-0e5dd17ccfa3","task_revision_id":"2","upstream_id":"","short_description":"BigCodeBench v0.1.0_hf f12070cd-8001-50c3-9511-0e5dd17ccfa3","config":"default","split":"v0.1.0_hf","body":"{\"code_prompt\":\"import socket\\nimport requests\\ndef task_func(host):\\n\",\"complete_prompt\":\"import socket\\nimport requests\\n\\ndef task_func(host):\\n    \\\"\\\"\\\"\\n    This function resolves the IP address of the given host and then uses the IP address \\n    to fetch geolocation information from the ipinfo.io API. The function is robust against\\n    various common errors, such as invalid hostnames, network issues, or problems with the \\n    geolocation service.\\n\\n    Parameters:\\n    host (str): The hostname to be resolved.\\n\\n    Returns:\\n    dict: A dictionary containing the IP address and geolocation information if successful.\\n\\n    Raises:\\n    ValueError: If 'host' is None or an empty string.\\n    ConnectionError: If there is a problem connecting to the geolocation service.\\n\\n    Example:\\n    >>> result = task_func('google.com')\\n    >>> 'ip_address' in result and 'geolocation' in result\\n    True\\n    >>> task_func('')\\n    Traceback (most recent call last):\\n       ...\\n    ValueError: Host must be a non-empty string.\\n    \\n    Requirements:\\n    - socket\\n    - requests\\n    \\\"\\\"\\\"\\n\",\"entry_point\":\"task_func\",\"instruct_prompt\":\"This function resolves the IP address of the given host and then uses the IP address to fetch geolocation information from the ipinfo.io API. The function is robust against various common errors, such as invalid hostnames, network issues, or problems with the geolocation service.\\nThe function should raise the exception for: ValueError: If 'host' is None or an empty string. ConnectionError: If there is a problem connecting to the geolocation service.\\nThe function should output with:\\n    dict: A dictionary containing the IP address and geolocation information if successful.\\nYou should write self-contained code starting with:\\n```\\nimport socket\\nimport requests\\ndef task_func(host):\\n```\",\"libs\":\"['requests', 'socket']\"}","display_format":"code","language":"","answer_status":"published","assets":[],"source_url":"https://bigcode-bench.github.io/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}