{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bigcodebench","formal_name":"BigCodeBench","introduction":"BigCodeBench poses 1,140 function-level tasks drawn across 139 libraries. It tests whether a model can compose several real APIs correctly rather than write one self-contained function.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://bigcode-bench.github.io/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"949e1fe8-382a-5b73-a3de-c40f4f675af1","task_key":"default--v0~2e1~2e0~5fhf--949e1fe8-382a-5b73-a3de-c40f4f675af1","task_revision_id":"1","upstream_id":"","short_description":"BigCodeBench v0.1.0_hf 949e1fe8-382a-5b73-a3de-c40f4f675af1","config":"default","split":"v0.1.0_hf","body":"{\"code_prompt\":\"import random\\nimport re\\ndef task_func(target_words, n_sentences, vocabulary):\\n\",\"complete_prompt\":\"import random\\nimport re\\n\\n\\ndef task_func(target_words, n_sentences, vocabulary):\\n    \\\"\\\"\\\"\\n    Generate sentences with spaces in certain target words replaced by underscores.\\n\\n    Parameters:\\n    - target_words (list of str): List of words/phrases where spaces should be replaced with underscores.\\n    - n_sentences (int):          Number of sentences to generate. Must not be negative.\\n    - vocabulary (list of str):   List of words to use for generating sentences. Must not be empty.\\n\\n    Returns:\\n    - list of str: A list of generated sentences in all lowercase, with specified words/phrases underscored.\\n\\n    Raises:\\n    - ValueError: If n_sentences is negative or if the vocabulary is empty.\\n\\n    Requirements:\\n    - random\\n    - re\\n\\n    Notes:\\n    - Each sentence is generated by randomly sampling 10 words with replacement from a vocabulary,\\n      then concatenating with a single whitespace. Then, if any words from the target_words list\\n      appear in these sentences, spaces within those words are replaced with underscores; here the\\n      modification is insensitive to the case of the letters.\\n    - The function returns the processed sentences as a list of all lowercase strings.\\n\\n    Examples:\\n    >>> random.seed(42)\\n    >>> task_func(['apple banana'], 1, ['apple', 'banana', 'cherry'])\\n    ['banana apple apple apple cherry cherry cherry apple_banana apple']\\n    >>> task_func(['Alice Charlie', 'ALICE BOB', 'aLiCe dAn'], 1, ['alice', 'bob', 'charlie', 'dan'])\\n    ['alice_charlie alice alice_charlie charlie alice_charlie dan alice']\\n    \\\"\\\"\\\"\\n\",\"entry_point\":\"task_func\",\"instruct_prompt\":\"Generate sentences with spaces in certain target words replaced by underscores.\\nNote that: Notes: Each sentence is generated by randomly sampling 10 words with replacement from a vocabulary, then concatenating with a single whitespace. Then, if any words from the target_words list appear in these sentences, spaces within those words are replaced with underscores; here the modification is insensitive to the case of the letters. The function returns the processed sentences as a list of all lowercase strings.\\nThe function should raise the exception for: ValueError: If n_sentences is negative or if the vocabulary is empty.\\nThe function should output with:\\n    list of str: A list of generated sentences in all lowercase, with specified words/phrases underscored.\\nYou should write self-contained code starting with:\\n```\\nimport random\\nimport re\\ndef task_func(target_words, n_sentences, vocabulary):\\n```\",\"libs\":\"['random', 're']\"}","display_format":"code","language":"","answer_status":"published","assets":[],"source_url":"https://bigcode-bench.github.io/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}