challenge-coding-004
Suite: coding
Finish: stop
Prompt
只输出 Python 代码,不要解释。实现 solve(capacity: int, ops: list[str]) -> list[int],模拟 LRU cache。ops 包含 "put key value" 和 "get key"。get 返回 value,不存在返回 -1。put 已存在 key 要更新并变成最近使用。容量为 0 时所有 get 都返回 -1。
Standard Answer / Scoring Expectation
Reference tests:
ops = ['put a 1', 'put b 2', 'get a', 'put c 3', 'get b', 'get a', 'get c']
assert solution.solve(2, ops) == [1, -1, 1, 3]
assert solution.solve(0, ['put a 1', 'get a']) == [-1]
assert solution.solve(1, ['put a 1', 'put a 2', 'get a']) == [2]
Mistake Analysis
Failure type: runtime_type_assumption
题目里的 cache key 是字符串,例如 `a`、`b`、`c`,但模型把 key 强行写成 `int(parts[1])`。测试一遇到 `put a 1` 就在解析 `a` 时抛出 ValueError。根因是模型擅自补了“key 是整数”的类型假设。
Scorer Details
{
"returncode": 1,
"stdout": "",
"stderr": "Traceback (most recent call last):\n File \"/var/folders/x_/4cht1kfj1_j31gdv5w01sgmw0000gn/T/dracobench-code-4wmtsl7u/test_solution.py\", line 4, in <module>\n assert solution.solve(2, ops) == [1, -1, 1, 3]\n ^^^^^^^^^^^^^^^^^^^^^^\n File \"/private/var/folders/x_/4cht1kfj1_j31gdv5w01sgmw0000gn/T/dracobench-code-4wmtsl7u/solution.py\", line 22, in solve\n key = int(parts[1])\n ^^^^^^^^^^^^^\nValueError: invalid literal for int() with base 10: 'a'\n"
}
Output
```python
from collections import OrderedDict
def solve(capacity: int, ops: list[str]) -> list[int]:
res = []
if capacity <= 0:
for op in ops:
parts = op.split()
if parts[0] == 'get':
res.append(-1)
return res
cache = OrderedDict()
for op in ops:
parts = op.split()
if parts[0] == 'get':
key = int(parts[1])
if key not in cache:
res.append(-1)
else:
cache.move_to_end(key)
res.append(cache[key])
elif parts[0] == 'put':
key = int(parts[1])
value = int(parts[2])
if key in cache:
cache[key] = value
cache.move_to_end(key)
else:
if len(cache) == capacity:
cache.popitem(last=False)
cache[key] = value
return res
```
challenge-coding-005
Suite: coding
Finish: incomplete
Prompt
只输出 Python 代码,不要解释。实现 solve(base: dict, patch: dict) -> dict,执行 JSON Merge Patch:patch 中值为 None 表示删除该 key;两个值都是 dict 时递归合并;其他值覆盖。不能修改输入对象。
Standard Answer / Scoring Expectation
Reference tests:
base = {'a': 1, 'b': {'x': 1, 'y': 2}, 'c': 3}
patch = {'b': {'y': None, 'z': 4}, 'c': None}
assert solution.solve(base, patch) == {'a': 1, 'b': {'x': 1, 'z': 4}}
assert base == {'a': 1, 'b': {'x': 1, 'y': 2}, 'c': 3}
assert solution.solve({'a': {'b': 1}}, {'a': 5}) == {'a': 5}
Mistake Analysis
Failure type: empty_output
模型正常结束但没有输出任何可评分正文,形成空输出。评分器无法命中答案或关键字段,因此失败。根因是答案生成阶段没有产出内容,而不是评分规则过严。
Scorer Details
{
"returncode": 1,
"stdout": "",
"stderr": "Traceback (most recent call last):\n File \"/var/folders/x_/4cht1kfj1_j31gdv5w01sgmw0000gn/T/dracobench-code-wjtmhlr7/test_solution.py\", line 5, in <module>\n assert solution.solve(base, patch) == {'a': 1, 'b': {'x': 1, 'z': 4}}\n ^^^^^^^^^^^^^^\nAttributeError: module 'solution' has no attribute 'solve'\n"
}
Output
challenge-coding-007
Suite: coding
Finish: incomplete
Prompt
只输出 Python 代码,不要解释。实现 solve(s: str) -> bool,检查括号 ()[]{} 是否配对。单引号或双引号内部的括号忽略;反斜杠会转义引号。输入字符串可以包含普通字符。
Standard Answer / Scoring Expectation
Reference tests:
assert solution.solve('{a:[(1)]}') is True
assert solution.solve("{'x': '(]'}") is True
assert solution.solve("['unterminated]") is False
assert solution.solve('([)]') is False
assert solution.solve("\"paren \\\" ) still quoted\"") is True
Mistake Analysis
Failure type: empty_output
模型正常结束但没有输出任何可评分正文,形成空输出。评分器无法命中答案或关键字段,因此失败。根因是答案生成阶段没有产出内容,而不是评分规则过严。
Scorer Details
{
"returncode": 1,
"stdout": "",
"stderr": "Traceback (most recent call last):\n File \"/var/folders/x_/4cht1kfj1_j31gdv5w01sgmw0000gn/T/dracobench-code-lu3xh3fg/test_solution.py\", line 3, in <module>\n assert solution.solve('{a:[(1)]}') is True\n ^^^^^^^^^^^^^^\nAttributeError: module 'solution' has no attribute 'solve'\n"
}
Output
challenge-debugging-001
Suite: debugging
Finish: stop
Prompt
下面代码在 total 之后再次读取数据会变成空列表。请指出根因,并给出最小修复。最后单独一行写 FIX: <修复后的第一行函数定义>。
```python
def summarize(items=iter([1, 2, 3])):
return sum(items)
```
Standard Answer / Scoring Expectation
Expected regex pattern:
FIX:\s*`?\s*def\s+summarize\(items=None\)\s*:
Mistake Analysis
Failure type: regex_miss
模型正确指出默认参数里的 iterator 会在第一次调用后被耗尽,但最后的 `FIX:` 行写成了 `def summarize(items=[1, 2, 3])`。当前 benchmark 期望的最小修复是把函数头改成 `def summarize(items=None):`,再在函数体内创建默认列表/迭代器。根因是模型修复了症状,但最终修复行没有采用标准的 None sentinel 写法,也引入了不推荐的可变默认参数形式。
Scorer Details
{
"pattern": "FIX:\\s*`?\\s*def\\s+summarize\\(items=None\\)\\s*:"
}
Output
### 根因
1. 核心问题是多余的`iter()`包装将列表转为了**单遍迭代器**:迭代器是有内部状态的一次性可迭代对象,`sum()`遍历会持续推进迭代器指针直到耗尽所有元素,计算完total后迭代器已无剩余元素,再次读取自然得到空结果。
2. 叠加Python默认参数的特性:默认参数在函数定义阶段仅求值一次,所有无参调用共享同一个默认对象,首次调用耗尽迭代器后,后续调用会持续复用这个已耗尽的迭代器,进一步导致跨调用返回空结果。
### 最小修复
移除默认值外层多余的`iter()`包装,直接使用列表`[1, 2, 3]`作为默认参数。列表是可重复迭代的序列容器,遍历时会生成临时迭代器访问元素,不会修改原列表本身的状态,既解决了同一次调用中sum后读取为空的问题,也避免了迭代器复用导致的跨调用异常。
FIX: def summarize(items=[1, 2, 3]):
challenge-coding-014
Suite: coding
Finish: incomplete
Prompt
只输出 Python 代码,不要解释。实现 solve(nums: list[int]) -> int。返回删除恰好一个元素后,剩余数组的最大连续子数组和。nums 长度至少为 2,可以全为负数。
Standard Answer / Scoring Expectation
Reference tests:
assert solution.solve([1,-2,0,3]) == 4
assert solution.solve([1,-2,-2,3]) == 3
assert solution.solve([-1,-1,-1]) == -1
assert solution.solve([2,1,-2,3]) == 6
Mistake Analysis
Failure type: empty_output
模型正常结束但没有输出任何可评分正文,形成空输出。评分器无法命中答案或关键字段,因此失败。根因是答案生成阶段没有产出内容,而不是评分规则过严。
Scorer Details
{
"returncode": 1,
"stdout": "",
"stderr": "Traceback (most recent call last):\n File \"/var/folders/x_/4cht1kfj1_j31gdv5w01sgmw0000gn/T/dracobench-code-2tl7e5mu/test_solution.py\", line 3, in <module>\n assert solution.solve([1,-2,0,3]) == 4\n ^^^^^^^^^^^^^^\nAttributeError: module 'solution' has no attribute 'solve'\n"
}
Output
challenge-coding-015
Suite: coding
Finish: incomplete
Prompt
只输出 Python 代码,不要解释。实现 solve(line: str) -> list[str],解析一行简化 CSV:逗号分隔;字段可以用双引号包裹;引号内逗号保留;引号内两个连续双引号表示一个双引号;空字段保留;空格是普通字符,不要自动 trim。
Standard Answer / Scoring Expectation
Reference tests:
assert solution.solve('a,"b,c",d') == ['a', 'b,c', 'd']
assert solution.solve('"a""b",,x') == ['a"b', '', 'x']
assert solution.solve(' a ," b " ') == [' a ', ' b ']
Mistake Analysis
Failure type: empty_output
模型正常结束但没有输出任何可评分正文,形成空输出。评分器无法命中答案或关键字段,因此失败。根因是答案生成阶段没有产出内容,而不是评分规则过严。
Scorer Details
{
"returncode": 1,
"stdout": "",
"stderr": "Traceback (most recent call last):\n File \"/var/folders/x_/4cht1kfj1_j31gdv5w01sgmw0000gn/T/dracobench-code-_v5kirzd/test_solution.py\", line 3, in <module>\n assert solution.solve('a,\"b,c\",d') == ['a', 'b,c', 'd']\n ^^^^^^^^^^^^^^\nAttributeError: module 'solution' has no attribute 'solve'\n"
}
Output