benchmarks.wiki

benchmarks.wiki / Public workspace

BigCodeBench / BigCodeBench v0.1.0_hf 27d6125f-c01a-5424-adb7-300202be76fb

Problem

Answer published by the source. Consult the official source to check your work against its answer.

complete prompt

import re
import nltk
import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer

# Make sure to download NLTK stopwords
nltk.download('stopwords')

# Define a regex pattern for matching all non-alphanumeric characters
ALPHANUMERIC = re.compile('[\W_]+')

# Load NLTK's list of English stop words
STOPWORDS = nltk.corpus.stopwords.words('english')


def task_func(texts):
    """
    Creates a document-term matrix (DTM) from a list of text documents using CountVectorizer from Scikit-learn.
    Texts are preprocessed by removing non-alphanumeric characters (excluding spaces),
    converting to lowercase, and excluding English stop words defined in NLTK.

    Parameters:
    - texts (list of str): The list of text documents to convert into a DTM.

    Returns:
    - pd.DataFrame: A DataFrame where rows represent documents and columns represent unique terms;
                    cell values indicate the frequency of a term in a document.

    Requirements:
    - re
    - nltk
    - pandas
    - sklearn.feature_extraction.text

    Example:
    >>> texts = ["Hello, world!", "Machine learning is great.", "Python is my favorite programming language."]
    >>> dtm = task_func(texts)
    """

instruct prompt

Creates a document-term matrix (DTM) from a list of text documents using CountVectorizer from Scikit-learn. Texts are preprocessed by removing non-alphanumeric characters (excluding spaces), converting to lowercase, and excluding English stop words defined in NLTK.
The function should output with:
    pd.DataFrame: A DataFrame where rows represent documents and columns represent unique terms;
    cell values indicate the frequency of a term in a document.
You should write self-contained code starting with:

Code

import re
import nltk
import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer
# Make sure to download NLTK stopwords
nltk.download('stopwords')
# Define a regex pattern for matching all non-alphanumeric characters
ALPHANUMERIC = re.compile('[\W_]+')
# Load NLTK's list of English stop words
STOPWORDS = nltk.corpus.stopwords.words('english')
def task_func(texts):

code prompt

Code

import re
import nltk
import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer
# Make sure to download NLTK stopwords
nltk.download('stopwords')
# Define a regex pattern for matching all non-alphanumeric characters
ALPHANUMERIC = re.compile('[\W_]+')
# Load NLTK's list of English stop words
STOPWORDS = nltk.corpus.stopwords.words('english')
def task_func(texts):

entry point

task_func

libs

  • nltk
  • pandas
  • re
  • sklearn

Discussion

Discussion

No discussion posts on this page yet. Share a minimal failing example, an algorithm with its complexity, or a reproducible command and result. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. Share a minimal failing example, an algorithm with its complexity, or a reproducible command and result. Use the posting template.

Source and history

task_id
27d6125f-c01a-5424-adb7-300202be76fb
task_key
default--v0~2e1~2e0~5fhf--27d6125f-c01a-5424-adb7-300202be76fb
task_revision_id
1
config / split
default / v0.1.0_hf

Official source

initial import