benchmarks.wiki / Public workspace

CRUXEval / Tasks

Choose a problem to read its original content, examples and discussion.

Option counts cover the whole public benchmark.

Clear filters

1–50 / 200 tasks

• default / test

  a"

  Answer published by the source

  Identifiers

  upstream_id

  sample_104

  task_id

  2a3508ad-fc4b-5de0-bb5d-c62fae17d6db

  task_key

  default--test--sample~5f104

• default / test

  987yhNSHAshd 93275yrgSgbgSshfbsfB"

  Answer published by the source

  Identifiers

  upstream_id

  sample_113

  task_id

  00c666f0-593e-5f8c-b1ec-207eb65df2eb

  task_key

  default--test--sample~5f113

• default / test

  a-.-.b", "-."

  Answer published by the source

  Identifiers

  upstream_id

  sample_114

  task_id

  1853d842-3c14-545d-a535-c70ed16349c4

  task_key

  default--test--sample~5f114

• default / test

  [1, 2, 3, 5, 8], 6

  Answer published by the source

  Identifiers

  upstream_id

  sample_123

  task_id

  1855b18c-3e12-5185-a66a-4581af5b08b0

  task_key

  default--test--sample~5f123

• default / test

  'i like you', ' ', -1

  Answer published by the source

  Identifiers

  upstream_id

  sample_124

  task_id

  0c05fa7e-8cb3-5d0e-b12e-fccda777d6f3

  task_key

  default--test--sample~5f124

• default / test

  'ONBPICJOHRHDJOSNCPNJ9ONTHBQCJ', 'J'

  Answer published by the source

  Identifiers

  upstream_id

  sample_129

  task_id

  17ab9770-4ee9-514e-a49d-7c367472518f

  task_key

  default--test--sample~5f129

• default / test

  CRUXEval test sample_135

  Answer published by the source

  Identifiers

  upstream_id

  sample_135

  task_id

  369a6900-2cb5-5bc0-a0bf-cd24d6b9fe35

  task_key

  default--test--sample~5f135

• default / test

  [3, 2, 0, 0, 2, 3]

  Answer published by the source

  Identifiers

  upstream_id

  sample_137

  task_id

  29995106-46ac-5914-b6ab-70032ebbaf8c

  task_key

  default--test--sample~5f137

• default / test

  daaX", "daaX"

  Answer published by the source

  Identifiers

  upstream_id

  sample_143

  task_id

  04d90c8d-cd68-59a0-8a27-0c896a174b35

  task_key

  default--test--sample~5f143

• default / test

  ''.join(['A'] * 20)

  Answer published by the source

  Identifiers

  upstream_id

  sample_152

  task_id

  2a88455d-ae70-54c3-896e-4a729e07c768

  task_key

  default--test--sample~5f152

• default / test

  'Hello There', '*'

  Answer published by the source

  Identifiers

  upstream_id

  sample_154

  task_id

  13e9d705-9886-5e74-b18f-258363adc843

  task_key

  default--test--sample~5f154

• default / test

  aboba 212 has 0 digits"

  Answer published by the source

  Identifiers

  upstream_id

  sample_157

  task_id

  25e26a59-501f-5679-97c9-8a5da5ee0880

  task_key

  default--test--sample~5f157

• default / test

  'zejrohaj', 'owc'

  Answer published by the source

  Identifiers

  upstream_id

  sample_16

  task_id

  03955d78-d104-52c4-a30c-f97fe7e66dc0

  task_key

  default--test--sample~5f16

• default / test

  'aaXXXXbbXXXXccXXXXde', 'QW'

  Answer published by the source

  Identifiers

  upstream_id

  sample_167

  task_id

  2f707549-5093-5838-925a-629555536a26

  task_key

  default--test--sample~5f167

• default / test

  'spain', 'b', 4

  Answer published by the source

  Identifiers

  upstream_id

  sample_168

  task_id

  2124b698-4a5c-56ac-b361-60a84f4bb6c3

  task_key

  default--test--sample~5f168

• default / test

  'taole'

  Answer published by the source

  Identifiers

  upstream_id

  sample_169

  task_id

  2f4c702e-d57e-5b01-9b3c-1f081a94832a

  task_key

  default--test--sample~5f169

• default / test

  [12, 0, 13, 4, 12], 12

  Answer published by the source

  Identifiers

  upstream_id

  sample_170

  task_id

  36a4af86-caa3-57b7-8175-74a67c6dd9fd

  task_key

  default--test--sample~5f170

• default / test

  [3, 4, 1, 2, 3]

  Answer published by the source

  Identifiers

  upstream_id

  sample_171

  task_id

  063ac647-45f4-5d7e-a916-a88060c1f4cf

  task_key

  default--test--sample~5f171

• default / test

  'Hey DUdE THis …

  Answer published by the source

  Identifiers

  upstream_id

  sample_177

  task_id

  032075d7-f47a-5d60-b2e9-f9492444d431

  task_key

  default--test--sample~5f177

• default / test

  [5, 4, 3, 2, 1, 0], 3

  Answer published by the source

  Identifiers

  upstream_id

  sample_18

  task_id

  29340a70-fc11-52d7-b9e4-e37dd5a40f2b

  task_key

  default--test--sample~5f18

• default / test

  'pvtso'

  Answer published by the source

  Identifiers

  upstream_id

  sample_186

  task_id

  302a484a-230e-5306-8de0-0ba981716ff4

  task_key

  default--test--sample~5f186

• default / test

  ", "sdasdnakjsda80"

  Answer published by the source

  Identifiers

  upstream_id

  sample_19

  task_id

  24d2b6c8-fcfa-59f0-b73d-22c0d4b17df7

  task_key

  default--test--sample~5f19

• default / test

  'Ohno'

  Answer published by the source

  Identifiers

  upstream_id

  sample_191

  task_id

  15d58171-c5cb-5950-9169-a75db7da65dc

  task_key

  default--test--sample~5f191

• default / test

  '!klcd!ma:ri', '!'

  Answer published by the source

  Identifiers

  upstream_id

  sample_192

  task_id

  1ed74de1-88fa-58ef-86f1-d868162ff0d0

  task_key

  default--test--sample~5f192

• default / test

  'tcmfsmj', 'cfj'

  Answer published by the source

  Identifiers

  upstream_id

  sample_198

  task_id

  19dd15b6-fcd5-5b3e-8105-14756ae118aa

  task_key

  default--test--sample~5f198

• default / test

  'mnmnj krupa...##!@#!@#…

  Answer published by the source

  Identifiers

  upstream_id

  sample_199

  task_id

  1abd4eac-6fc3-53b3-aae1-c6d1d91bcd29

  task_key

  default--test--sample~5f199

• default / test

  'hbtofdeiequ'

  Answer published by the source

  Identifiers

  upstream_id

  sample_2

  task_id

  1d99572a-d5bd-5a3d-bdf5-1b5ab724e8d8

  task_key

  default--test--sample~5f2

• default / test

  000

  Answer published by the source

  Identifiers

  upstream_id

  sample_22

  task_id

  291ed48d-e844-526b-bc02-881ba02dbc19

  task_key

  default--test--sample~5f22

• default / test

  [35, 45, 3, 61, 39, 27, 47], 0

  Answer published by the source

  Identifiers

  upstream_id

  sample_24

  task_id

  093ffcb8-0c73-5ecd-98c0-302aa787c100

  task_key

  default--test--sample~5f24

• default / test

  '-123 …

  Answer published by the source

  Identifiers

  upstream_id

  sample_29

  task_id

  08e8c662-fcdf-5eff-93b7-ea5d075392e3

  task_key

  default--test--sample~5f29

• default / test

  [1, 2, 3, 7, 7, 6, 8, 4, 1, 2, 3, 5, 1, 3, 21, 1, 3], 3, 1

  Answer published by the source

  Identifiers

  upstream_id

  sample_34

  task_id

  08074946-7923-523b-9334-788575261926

  task_key

  default--test--sample~5f34

• default / test

  B ", [" bBb ", " BaB ", " bB", " bBbB ", " bbb"]

  Answer published by the source

  Identifiers

  upstream_id

  sample_35

  task_id

  215c3d37-4505-527f-83cd-c793e25705e8

  task_key

  default--test--sample~5f35

• default / test

  [4, 3, 2, 1, 2, -1, 4, 2]

  Answer published by the source

  Identifiers

  upstream_id

  sample_42

  task_id

  06f1bfca-c637-5049-94be-d5b8bd95833a

  task_key

  default--test--sample~5f42

• default / test

  6 ** 2"

  Answer published by the source

  Identifiers

  upstream_id

  sample_43

  task_id

  157ba45a-8f4b-5db9-80b8-c61a66138a46

  task_key

  default--test--sample~5f43

• default / test

  'za1fd1as8f7afasdfam97adfa', '7'

  Answer published by the source

  Identifiers

  upstream_id

  sample_45

  task_id

  073117d9-ca5b-5e32-90bf-e3ed55a88155

  task_key

  default--test--sample~5f45

• default / test

  '816'

  Answer published by the source

  Identifiers

  upstream_id

  sample_49

  task_id

  317d00f7-3e0b-5e79-b284-a08bb83ff090

  task_key

  default--test--sample~5f49

• default / test

  '1z1z1'

  Answer published by the source

  Identifiers

  upstream_id

  sample_56

  task_id

  08a23766-25cb-5558-9a3b-f55bdd8a9cb3

  task_key

  default--test--sample~5f56

• default / test

  'hi '

  Answer published by the source

  Identifiers

  upstream_id

  sample_59

  task_id

  011ec5ba-1d5c-595a-a3a2-dc87191ac07b

  task_key

  default--test--sample~5f59

• default / test

  "

  Answer published by the source

  Identifiers

  upstream_id

  sample_61

  task_id

  15000693-c923-5182-97d8-0dd8d62a12bf

  task_key

  default--test--sample~5f61

• default / test

  []

  Answer published by the source

  Identifiers

  upstream_id

  sample_7

  task_id

  09768e2d-9519-571e-95f0-cac9cbd08958

  task_key

  default--test--sample~5f7

• default / test

  '999893767522480'

  Answer published by the source

  Identifiers

  upstream_id

  sample_70

  task_id

  05c7c14c-5e3e-54cf-81b5-d6cd2298aaef

  task_key

  default--test--sample~5f70

• default / test

  1: 2, 3: 4, 5: 6, 7: 8, 9: 10, 1

  Answer published by the source

  Identifiers

  upstream_id

  sample_71

  task_id

  0ab4b131-19d5-5baa-87ee-4a99d80b7ccb

  task_key

  default--test--sample~5f71

• default / test

  '99'

  Answer published by the source

  Identifiers

  upstream_id

  sample_72

  task_id

  0b297728-7597-5ac9-81f0-bf3a3bae7e0e

  task_key

  default--test--sample~5f72

• default / test

  [44, 34, 23, 82, 24, 11, 63, 99], 4, 15

  Answer published by the source

  Identifiers

  upstream_id

  sample_74

  task_id

  30df8e7a-6c1f-5619-87a0-0d1cf99d26c9

  task_key

  default--test--sample~5f74

• default / test

  "Bulls": 23, "White Sox": 45, "Bulls"

  Answer published by the source

  Identifiers

  upstream_id

  sample_81

  task_id

  1f8e4947-8a30-5082-b8e8-0a892a9047a7

  task_key

  default--test--sample~5f81

• default / test

  'qq0tt'

  Answer published by the source

  Identifiers

  upstream_id

  sample_83

  task_id

  24dae924-b06c-5a2c-8ba0-7c5ecec00326

  task_key

  default--test--sample~5f83

• default / test

  he", "hello"

  Answer published by the source

  Identifiers

  upstream_id

  sample_88

  task_id

  15869254-db3e-5a8f-b491-d94047ff3367

  task_key

  default--test--sample~5f88

• default / test

  [[1, 2, 3], [], [1, 2, 3]]

  Answer published by the source

  Identifiers

  upstream_id

  sample_90

  task_id

  2d1a7842-59fe-57db-82aa-8050d0545f5c

  task_key

  default--test--sample~5f90

• default / test

  'AAA': 'fr'

  Answer published by the source

  Identifiers

  upstream_id

  sample_95

  task_id

  2c1b919c-82e3-5619-a64c-764a2e1076b8

  task_key

  default--test--sample~5f95

• default / test

  'aa+++bb', '+', 1

  Answer published by the source

  Identifiers

  upstream_id

  sample_99

  task_id

  37dfe3ee-fbe8-55d4-b1a0-5f72bbd80674

  task_key

  default--test--sample~5f99