{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bigcodebench","formal_name":"BigCodeBench","introduction":"BigCodeBench poses 1,140 function-level tasks drawn across 139 libraries. It tests whether a model can compose several real APIs correctly rather than write one self-contained function.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://bigcode-bench.github.io/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4b2c1b2e-2728-5ada-96fb-72786c3ae12e","task_key":"default--v0~2e1~2e0~5fhf--4b2c1b2e-2728-5ada-96fb-72786c3ae12e","task_revision_id":"1","upstream_id":"","short_description":"BigCodeBench v0.1.0_hf 4b2c1b2e-2728-5ada-96fb-72786c3ae12e","config":"default","split":"v0.1.0_hf","body":"{\"code_prompt\":\"import pandas as pd\\nfrom sklearn.model_selection import train_test_split\\nfrom sklearn.linear_model import LinearRegression\\nimport numpy as np\\ndef task_func(data):\\n\",\"complete_prompt\":\"import pandas as pd\\nfrom sklearn.model_selection import train_test_split\\nfrom sklearn.linear_model import LinearRegression\\nimport numpy as np\\n\\ndef task_func(data):\\n    ''' \\n    Train a simple linear regression model based on the given data and evaluate the model by calculating the mean square error. The data should be structured with 'Hours' as independent variables and 'Scores' as dependent variables.\\n    The function set the random set when dividing the train and test data to 42 and the test set size is 0.2\\n\\n    Parameters:\\n    - data (dict): The dictionary with keys 'Hours' and 'Scores', representing study hours and respective scores.\\n\\n    Returns:\\n    float: The mean squared error between the actual scores and predicted scores based on the test split.\\n\\n    Requirements:\\n    - pandas\\n    - sklearn.model_selection.train_test_split\\n    - sklearn.linear_model.LinearRegression\\n    - numpy\\n\\n    Example:\\n    >>> task_func({'Hours': [10, 20, 40], 'Scores': [90, 80, 70]})\\n    25.0\\n    '''\\n\",\"entry_point\":\"task_func\",\"instruct_prompt\":\"Train a simple linear regression model based on the given data and evaluate the model by calculating the mean square error. The data should be structured with 'Hours' as independent variables and 'Scores' as dependent variables. The function set the random set when dividing the train and test data to 42 and the test set size is 0.2\\nThe function should output with:\\n    float: The mean squared error between the actual scores and predicted scores based on the test split.\\nYou should write self-contained code starting with:\\n```\\nimport pandas as pd\\nfrom sklearn.model_selection import train_test_split\\nfrom sklearn.linear_model import LinearRegression\\nimport numpy as np\\ndef task_func(data):\\n```\",\"libs\":\"['pandas', 'numpy', 'sklearn']\"}","display_format":"code","language":"","answer_status":"published","assets":[],"source_url":"https://bigcode-bench.github.io/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}