mirror of
https://github.com/dogkeeper886/ollama37.git
synced 2025-12-22 13:46:59 +00:00
2c5094db92
Problem: Tests used `docker compose logs --since=5m` which caused:
- Log overlap between tests
- Logs from previous tests included
- Missing logs if test exceeded 5 minutes
Solution:
- New LogCollector class runs `docker compose logs --follow`
- Marks test start/end boundaries
- Writes test-specific logs to /tmp/test-{testId}-logs.txt
- Test steps access via TEST_ID environment variable
Changes:
- tests/src/log-collector.ts: New LogCollector class
- tests/src/executor.ts: Integrate LogCollector, set TEST_ID env
- tests/src/cli.ts: Start/stop LogCollector for runtime/inference
- All test cases: Use log collector with fallback to docker compose
Also updated docs/CICD.md with:
- Test runner CLI documentation
- Judge modes (simple, llm, dual)
- Log collector integration
- Updated test case list (12b, 27b models)
- Model unload strategy
- Troubleshooting guide
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
67 lines
2.0 KiB
YAML
67 lines
2.0 KiB
YAML
id: TC-RUNTIME-001
|
|
name: Container Startup
|
|
suite: runtime
|
|
priority: 1
|
|
timeout: 120000
|
|
|
|
dependencies:
|
|
- TC-BUILD-002
|
|
|
|
steps:
|
|
- name: Stop existing container
|
|
command: cd docker && docker compose down 2>/dev/null || true
|
|
|
|
- name: Start container with GPU
|
|
command: cd docker && docker compose up -d
|
|
|
|
- name: Wait for startup
|
|
command: sleep 15
|
|
|
|
- name: Check container status
|
|
command: cd docker && docker compose ps
|
|
|
|
- name: Capture startup logs
|
|
command: |
|
|
# Use log collector file if available, fallback to docker compose logs
|
|
if [ -f "/tmp/test-${TEST_ID}-logs.txt" ]; then
|
|
head -100 /tmp/test-${TEST_ID}-logs.txt
|
|
else
|
|
cd docker && docker compose logs 2>&1 | head -100
|
|
fi
|
|
|
|
- name: Check for startup errors in logs
|
|
command: |
|
|
# Use log collector file if available, fallback to docker compose logs
|
|
if [ -f "/tmp/test-${TEST_ID}-logs.txt" ]; then
|
|
LOGS=$(cat /tmp/test-${TEST_ID}-logs.txt)
|
|
else
|
|
LOGS=$(cd docker && docker compose logs 2>&1)
|
|
fi
|
|
|
|
# Check for critical errors
|
|
if echo "$LOGS" | grep -qE "(level=ERROR|CUBLAS_STATUS_|CUDA error|cudaMalloc failed)"; then
|
|
echo "CRITICAL ERRORS FOUND IN STARTUP LOGS:"
|
|
echo "$LOGS" | grep -E "(level=ERROR|CUBLAS_STATUS_|CUDA error|cudaMalloc failed)"
|
|
exit 1
|
|
fi
|
|
|
|
# Check for CPU-only fallback (GPU not detected)
|
|
if echo "$LOGS" | grep -q "id=cpu library=cpu"; then
|
|
echo "ERROR: Ollama fell back to CPU-only mode"
|
|
exit 1
|
|
fi
|
|
|
|
echo "SUCCESS: No critical errors in startup logs"
|
|
|
|
criteria: |
|
|
The ollama37 container should start successfully with GPU access.
|
|
|
|
Expected:
|
|
- Container starts without errors
|
|
- docker compose ps shows container in "Up" state
|
|
- No "Exited" or "Restarting" status
|
|
- No critical errors in logs (level=ERROR, CUBLAS_STATUS_, CUDA error)
|
|
- No CPU-only fallback (id=cpu library=cpu)
|
|
|
|
Accept startup warnings (flash attention not supported is OK). Container should be running.
|