Visit Contact us Support Us

NAAI member， Professor Li Hui from Peking University and several other teams have joined forces with the Tencent Platform Security Department to build a secure code generation testing platform：AICGSecEval

Information

Tencent Launches “AICGSecEval” — Industry’s First Project-Level AI Code Security Benchmark 


 As AI-powered code generation continues to reshape software development, security vulnerabilities—such as hard-to-spot logic flaws or missing validation—have raised significant concerns. Responding to this, Tencent Security Platform’s WuKong Code Security Team today announced the open-source release of AICGSecEval (AI Code Generation Security Evaluation), the industry’s first project-level evaluation framework designed to assess the security of AI-generated code in realistic settings (GitHub).

 What Makes AICGSecEval Unique

  • Repository-level testing: Unlike typical benchmarks focused on single functions or files, AICGSecEval simulates real-world development by evaluating AI-generated code across complete GitHub repositories (GitHub).

  • Expert‑labeled CVE scenarios: Built around authentic vulnerability cases (e.g., XSS, SQL injection, path traversal), the framework uses mutated seed data to avoid biases while preserving evaluation rigor (GitHub).

  • Multi-dimensional scoring: Assesses code on security accuracy (CVE detection), integration success (SAST-based quality), and stability across multiple generation runs (GitHub).

  • Transparent, reproducible dataset: All data, evaluation protocols, and code are open-source, ensuring the results are publicly verifiable (aicgseceval.tencent.com).

Academic Collaboration

The initiative is a joint effort led by Tencent, with development contributions from several leading university labs:

  • Fudan University’s System Software & Security Lab

  • Peking University team led by Prof. Li Hui

  • Shanghai Jiao Tong University’s Network & System Security Institute

  • Tsinghua University team led by Prof. Yujiu Yang

  • Zhejiang University team led by Asst. Prof. Ziming Zhao (Reuters, GitHub)

Each partner played a key role in designing threat scenarios, identifying vulnerabilities, refining scan rules, and validating results—ensuring the framework is grounded in academic rigor and real-world relevance.

 Open Source & Community Engagement

  • The code is available at github.com/Tencent/AICGSecEval and already has 112 stars and 5 forks (GitHub). Developers are encouraged to submit issues, pull requests, new test cases, and enhancements.

  • The official website aicgseceval.tencent.com offers benchmark rules, running guides, and model leaderboards (aicgseceval.tencent.com).

  • Community feedback is gathered through a user-survey at Tencent’s documentation portal, with incentives for detailed responses (GitHub).

Why It Matters

AICGSecEval marks a significant leap forward in AI code security:

  • Practical validation: It equips teams to test AI-generated code within production-like environments—essential for DevSecOps pipelines.

  • Standardization: Offers a unified, transparent metric for evaluating compliance, tightening the overall quality and safety of AI-generated artifacts.

  • Ecosystem engagement: By opening development to both academia and community contributors, it creates a collaborative model for continuously elevating AI coding standards.

 Get Involved


AICGSecEval is more than just a benchmark—it’s a collaborative leap toward safer, more reliable AI programming. Join the effort to fortify the next generation of intelligent code!