Secure Application Development: What an AI-Ready Security Posture Looks Like
AI writes more of your code, and attackers automate more of their work. Here is how to build security into the pipeline so every change is checked before it ships.

Secure application development is the practice of building security into every stage of delivery, from design to release, so that each change is checked before it ships rather than after an incident. When AI writes part of the code, that means treating every AI-written change as untrusted until your pipeline has checked it, and running those checks on every merge rather than in an annual audit. In practice that is five habits: scan code, dependencies and secrets in CI; keep human review on the paths that matter (authentication, access control, payments, data handling); know every component you ship; give tokens and agents the least access that works; and test the running product continuously. The urgency is new, not the playbook. In Veracode’s October 2026 GenAI Code Security update, AI models still wrote vulnerable code in 38% of security-relevant tasks. Meanwhile attackers are automating their side of the work, as our companion piece on whether AI can hack your software documents.
This guide is the defensive half of that story, written for CTOs and engineering leads at small and mid-sized software companies: the controls, the order to add them, and how they map to CISA Secure by Design, the NIST Secure Software Development Framework (SSDF) and the OWASP Top 10 for LLM Applications.
What secure application development means when AI writes the code
AI coding assistants do not change what secure application development is; they change the volume and source of changes, which is why the checks have to move from people’s calendars into the pipeline.
The old model assumed a human wrote each line and understood it, review caught most mistakes, and an annual penetration test caught some of the rest. AI-written changes may come from an assistant that has not been given your threat model, data classification or contractual requirements. Give it the relevant context, then verify its output independently.
An AI-ready posture rests on four ideas:
- AI output is untrusted input. It gets the same treatment as a pull request from a contributor you have never met: automated checks first, human judgment where it counts.
- The pipeline is the control point. Policies in a wiki do not scale to AI-speed delivery. Checks that run on every commit do.
- Your build chain is attack surface. Dependencies, CI runners, tokens and coding-assistant configuration are now targets in their own right.
- Verification beats generation. Finding and fixing are cheap with AI. Knowing that a fix is correct is the scarce skill, and the one your process has to protect.
The same controls also produce the evidence that enterprise security reviews and the EU Cyber Resilience Act ask for.
Secure design and threat modeling come first
Pipeline checks catch flaws in code that already exists. Secure design stops the flaws that would otherwise be built in, and it starts before the first line is written, by a person or a model. For each feature or release, write a short threat model: what the feature protects, who could attack it and how, and what the damage would be. Turn the answers into security requirements on the ticket, such as who may see the data, what must be logged and what must be encrypted, and settle authentication, authorization and encryption choices in design review rather than in code review.
This matters more with AI in the loop. An assistant given the threat model and the requirements as context writes against them, and a reviewer has something concrete to check its output against.
Is AI-generated code safe? What the newest data shows
AI-generated code is getting safer, but it is not safe by default. The best models in Veracode’s October 2026 test still failed more than one security-relevant task in four, and other recent research shows that AI-written security fixes often look right without being right. Both findings point to the same answer: generation is fine, unverified merging is not.
Veracode: better on easy flaws, flat on hard ones
Veracode has tested AI-generated code since July 2025, asking models to complete coding tasks with known potential for vulnerabilities and no security-specific instructions. Its October 29, 2026 update tested 18 models across Python, C#, JavaScript and Java. The average security pass rate rose from 56% to 62%, and the share of tasks producing vulnerable code fell from 44% to 38%: the first meaningful improvement in the series. The top two models tied at 73%.
The detail matters more than the headline. Most of the gain came from cross-site scripting, the flaw type that pattern recognition fixes most easily, while log injection, which requires reasoning about data flow across functions, stayed flat at a 13% pass rate. Models are learning to avoid the obvious mistakes and still miss the ones that need context.
AI fixes that look right but are not
Two studies from the last few months examine the step after detection: fixing.
In a study published on August 6, 2026, 1Password’s Off-by-1 Labs had frontier models patch six recently disclosed vulnerabilities requiring complex fixes and graded 6,080 of the patches. Across prompting conditions that included correct and incorrect vulnerability guidance, only 26.0% resolved the vulnerability without materially changing application behavior or introducing new problems, and 53.9% left the vulnerability unresolved, introduced a new one, or both.
The human side is no more comforting. As IT Pro reported on November 4, 2026, SecureFlag analyzed 805,663 attempts by developers to fix vulnerabilities on its training platform over two years. A quarter of the fixes were incorrect or incomplete. Broken access control and software and data integrity failures were the hardest to fix, with three in ten attempts on each failing. SecureFlag’s CTO warned that developers who cannot fix a vulnerability themselves cannot reliably verify an AI’s fix either.
Put together, the data supports one operating rule: let AI write and suggest, but make the pipeline and a qualified reviewer decide what ships.
How to secure AI-generated code before it merges
To secure AI-generated code, treat it exactly like code from an unknown contributor: run automated security checks on every pull request, require human review on high-risk paths, verify every dependency it introduces, and keep the assistant away from secrets and production data.
- Scan every pull request automatically. Static application security testing (SAST) reads the code for injection, unsafe deserialization, weak cryptography and similar flaws, and engineers see findings in minutes.
- Define the paths that always get human review. Authentication, authorization, payments, file uploads, data exports, cryptography and anything touching personal data. A reviewer who understands the threat model signs off, whoever (or whatever) wrote the code.
- Verify what the assistant imports. Models suggest packages that are outdated, abandoned or occasionally nonexistent, and a nonexistent name is an invitation for someone to register it.
- Write the tests in the same pass. Ask for tests alongside the feature, including negative tests for access control, and review them as carefully as the code.
- Keep assistants out of production. No production credentials in the developer environment, no real user data in prompts, and no auto-execution of terminal commands without approval.
- Run a security review pass before release. Asking a model to review a feature as an attacker would is a cheap extra layer that gives the human reviewer a head start.
Several of these controls are part of our AI-assisted delivery on a pet adoption and rehoming marketplace we develop with a Forward Deployed Engineer working with Claude. On the marketplace’s messaging layer, the engineer ran a security review with Claude Code on every feature before it shipped. Tests are written alongside each change and run in CI. The engineer reviews everything Claude produces and decides what ships, and the founder verifies each feature on staging before release. Claude works on the code, designs and documents, and never touches real user data.
CI/CD pipeline security: a check on every change
CI/CD pipeline security has two jobs: run security checks on every change that passes through the pipeline, and protect the pipeline itself, because a compromised build system ships malware with your signature on it. Recent attacks have aimed squarely at the second job.
The checks, layered from cheapest to most thorough
| Check | What it catches | When it runs |
|---|---|---|
| Secret scanning | API keys, tokens and passwords committed to code or history | Every commit, plus a pre-commit hook |
| Software composition analysis (SCA) | Known-vulnerable and malicious dependencies, license problems | Every commit and on a daily schedule |
| Static analysis (SAST) | Injection, unsafe APIs, weak crypto, risky patterns in your own code | Every pull request |
| Infrastructure-as-code scanning | Public buckets, open security groups, over-broad IAM roles | Every pull request that touches infrastructure |
| Dynamic testing (DAST) | Flaws visible only in the running app: auth gaps, misconfigured headers, exposed endpoints | Against staging on every release |
Start every new check in report-only mode, tune out the noise, then make new critical findings block the merge. A gate that fires constantly gets switched off.
Protect the pipeline, including AI tool configuration
Developer tooling has become a way in. On October 27, 2026, CSO Online reported JFrog’s research into 42 compromised third-party GitHub repositories whose malicious payloads were large enough to evade GitHub’s own search. The victim repositories contained execution hooks for AI coding assistants and IDEs, and JFrog advised auditing repositories for files such as .claude/settings.json, .cursor/mcp.json and .vscode/tasks.json, alongside rotating every developer and CI credential.
The practical controls:
- Review AI assistant and IDE configuration like code. Hooks, MCP server definitions and task files can execute commands, so changes go through pull requests and code owners.
- Pin and review CI workflows. Pin third-party actions to a commit hash and apply branch protection to everyone, administrators included.
- Give each job only the permissions it needs. A test job does not need publishing rights.
- Separate triggering a release from approving it. Different people trigger and approve releases, and that rule is never relaxed for convenience.
Software supply chain security: know what you ship
Software supply chain security means knowing every component in your product, controlling how new components get in, and being able to respond within hours when one of them turns out to be vulnerable or malicious. A software bill of materials (SBOM) is the foundation, because you cannot patch, report or reassure a customer about a library you do not know you ship.
CISA’s Secure by Design joint guidance asks manufacturers to build and maintain an SBOM for each product, request the same data from their suppliers and make SBOMs available to customers. Build tooling can emit one in CycloneDX or SPDX format on every build; the value comes from matching it against new advisories daily and answering a customer’s “are we affected?” in minutes.
For manufacturers placing covered products with digital elements on the EU market, the Cyber Resilience Act introduces security and reporting obligations. Reporting obligations have applied since September 11, 2026, and the main obligations apply from December 11, 2027. The Commission’s summary of the legal text sets deadlines once a manufacturer becomes aware of an actively exploited vulnerability in a covered product: an early warning within 24 hours, a notification within 72 hours and a final report within 14 days of a corrective or mitigating measure becoming available.
The Commission’s implementation FAQ explains that standalone SaaS is generally outside the CRA’s scope. Remote data processing can be covered when its software is designed and developed by or under the responsibility of a product’s manufacturer, and its absence would prevent the product from performing one of its functions.
The Act also requires manufacturers of covered products to exercise due diligence on the third-party components they integrate. The FAQ lists software composition analysis and reviewing a component’s SBOM among the examples. A 24-hour reporting window is very hard to meet without an inventory you can query.
For day-to-day dependency hygiene:
- Commit lockfiles and pin versions, so a build tomorrow uses the same code as the build you tested today.
- Delay brand-new releases. The malicious LiteLLM release described below was live on PyPI for about two hours before it was frozen; container images remained available longer. A short cooling-off period can help keep such releases out of your builds.
- Restrict install-time scripts where your package manager allows it, and treat any machine that installed a known-malicious version as compromised.
Secrets management: what the LiteLLM incident teaches
Secrets management is the discipline of keeping credentials out of code, giving each one the narrowest scope and shortest life that works, and revoking it the moment it may have been exposed. The most instructive recent example comes from a team that published a detailed account of what went wrong.
On October 21, 2026, a compromised version of LiteLLM, an open-source gateway for AI models, was published to PyPI and its container registries. In its account of the attack chain, published two days later, LiteLLM traced the incident back to May:
- A key leaked into a public repository. In May, a configuration file containing an administrative key to an internal deployment was committed to a public repository. It was deleted about three hours later, but the original commit stayed in the public history.
- Nobody revoked it. The key was flagged for revocation, but because of human error it was still valid more than four months later.
- An agent platform held an owner-level token. The attacker used the key to take over an internal platform for AI agents that open pull requests. That platform was configured with the GitHub command-line OAuth token of an organization owner, not a token scoped to specific repositories.
- CI secrets were harvested. With owner rights, the attacker pushed workflow changes to exfiltrate the secrets available to CI, collecting tokens belonging to two maintainers.
- A relaxed release control did the rest. A restriction that normally required approval from an account tied to a physical device had been relaxed that week to allow faster releases. The attacker used one stolen token to trigger the release and another to approve it.
Each step maps to a control any team can put in place:
- Deleting is not revoking. Treat any secret that touched a repository, a log or a chat as exposed and rotate it immediately; Git history and forks keep copies.
- Use a secrets manager, not files. Applications and pipelines fetch credentials at runtime from a vault or the cloud provider’s secret store.
- Scope every token. Never put a personal or owner-level credential inside an automated system, and least of all inside an AI agent, which acts on whatever access it is given.
- Prefer short-lived credentials. Workload identity federation (OIDC) gives CI jobs cloud credentials that expire in minutes instead of long-lived keys.
- Keep release approvals strict. Convenience exceptions are exactly what an attacker with stolen tokens looks for.
From annual pentests to continuous, AI-assisted testing
An annual penetration test describes your product on the day it was tested. A team shipping weekly changes its attack surface dozens of times before the next test, so testing has to become continuous too, with point-in-time tests as a deep check on top of an always-on baseline.
A continuous testing program has three layers:
- Automated checks in the pipeline, described above, catch the common flaw classes on every change.
- Scheduled automated testing of the running product, such as dynamic scans of staging and production-like environments, attack-surface monitoring of what is exposed to the internet, and AI-assisted triage that groups and prioritizes findings.
- Periodic expert testing focused on what automation misses: business logic, chained exploits and broken access control. In SecureFlag’s analysis, three in ten developer attempts to fix broken access control failed.
AI belongs on the defensive side of this loop, and the economics support it. IBM’s 2025 Cost of a Data Breach report found that organizations using AI and automation extensively throughout security operations saved an average of $1.9 million in breach costs and shortened the breach lifecycle by an average of 80 days. The 2026 edition put the saving at almost $2 million, while noting that one in four organizations had still not adopted these tools in security operations.
AI is good at finding candidate issues and drafting fixes, but only 26.0% of the AI-generated patches in 1Password’s study were clean fixes, so someone still has to confirm that each fix is correct. Keep a person accountable for closing each finding, require a test that proves the fix, and measure how quickly real findings get closed rather than how many you generate.
Securing the AI features you ship
If your product includes AI features, such as a chat assistant, retrieval over customer documents or an agent that takes actions, those features need their own threat model. The OWASP Top 10 for LLM Applications 2026 is the standard checklist, and four of its entries deserve attention from any team shipping AI to customers.
- Prompt injection (LLM01). Users, documents and web pages can carry instructions that change the model’s behavior. Treat all model input as untrusted, separate instructions from data, and never let model output alone authorize a sensitive action.
- Sensitive information disclosure (LLM02). Models can reveal data from their context, training or connected systems. Filter what goes in, filter what comes out, and keep retrieval scoped to what the current user may see.
- Excessive agency (LLM03). An agent with broad permissions and no confirmation step can do broad damage. Give agents the narrowest tools and credentials that work, require human approval for irreversible actions, and log every action. The LiteLLM incident shows what an over-privileged agent credential can reach.
- Improper output handling (LLM10). Model output inserted into a web page, a database query or a shell command is an injection risk like any other input. Validate and encode it.
Guardrails, evaluation and observability separate a demo agent from a production one, as our guide to why most AI agents never reach production explains. Design the agent’s permissions as if its instructions will one day come from an attacker.
Secure software development frameworks: how CISA, NIST and OWASP fit together
Three frameworks cover most of what a software company needs: CISA Secure by Design sets the principles, the NIST SSDF lists the development practices, and OWASP provides the technical standards for specific risks. You do not need to adopt all three formally, only a set of controls that would hold up against each.
CISA Secure by Design. The joint guidance from CISA, the NSA, the FBI and international partners rests on three principles: take ownership of customer security outcomes, embrace radical transparency and accountability, and lead from the top. Secure by design means customer security is a core business goal before development starts; secure by default means products are safe out of the box at no extra charge. Its recommendations include eliminating default passwords, requiring MFA for privileged users and publishing SBOMs.
NIST SSDF. The Secure Software Development Framework (SP 800-218, version 1.1) organizes its practices into four groups: Prepare the Organization, Protect the Software, Produce Well-Secured Software and Respond to Vulnerabilities. Version 1.2 has been in draft since December 2025, and SP 800-218A adds practices specific to generative AI.
OWASP. For the application itself, the OWASP Top 10 names the most critical web application security risks, and the Application Security Verification Standard (ASVS) is a broader standard of testable security requirements for designing, building and testing web applications, which makes it usable as acceptance criteria. Together they set the coding baseline: validate every input, encode every output and deny access by default. The OWASP Top 10 for LLM Applications covers the AI features you ship.
| Control in this guide | CISA Secure by Design | NIST SSDF group | OWASP |
|---|---|---|---|
| Scanning AI-generated code on every pull request | Eliminate vulnerability classes | Produce Well-Secured Software | OWASP Top 10 and ASVS |
| SBOM and dependency controls | Publish SBOMs | Protect the Software | Supply chain (LLM04) for AI components |
| Secrets management and scoped tokens | Secure by default | Protect the Software | Sensitive information disclosure (LLM02) |
| Continuous testing and fast remediation | Take ownership of customer outcomes | Respond to Vulnerabilities | ASVS requirements as test criteria |
| Least privilege for agents and AI features | Secure by default | Produce Well-Secured Software | Excessive agency (LLM03) |
Where to start: a sequence for small and mid-sized teams
Doing everything at once is how security programs stall. Keep threat modeling and human review of high-risk changes in place from day one, then add the tooling in this order:
- Weeks one and two: secrets. Turn on secret scanning, rotate anything exposed, move credentials into a secrets manager and replace broad tokens with scoped ones.
- Weeks three and four: dependencies. Generate an SBOM on every build, enable dependency and malicious-package scanning, and commit lockfiles everywhere.
- Month two: pull request checks. Add SAST in report-only mode, formalize review ownership for high-risk paths, and protect branches and workflows.
- Month three: the running product. Add dynamic testing against staging, set patch targets by severity and measure time to close.
- Ongoing: AI features and evidence. Review and update threat models for AI features against the OWASP list and keep the artifacts that security reviews and regulators ask for.
If your product is a short-lived prototype with no customer data, the full program is more than you need: secret scanning, dependency scanning and careful review of authentication cover most of the risk until real users arrive.
How we approach security in AI-assisted delivery
We treat security as part of how software ships, not a phase that follows release. When we develop or extend a product through our custom software development work, security checks run inside the automated build and deployment pipeline, a practice called DevSecOps. When a team already has a product and needs its release path hardened, our DevOps consulting builds the agreed security checks, such as vulnerability scanning, compliance checks and change approvals, into your release path. We design that path around the existing tools that meet its requirements, including your CI, cloud and ticketing tools. Each job gets least-privilege access, and your engineers get the runbooks they need to operate the path.
On the marketplace engagement described earlier, AI-assisted delivery runs on several of the controls in this guide: AI output reviewed by the engineer who owns the change, tests written alongside features, security review before release, and no AI access to real user data. Our QA practice includes ISTQB-certified engineers, and unicrew is itself certified to ISO 27001:2022 and ISO 9001:2015, audited by Quay Audit UK.
For SaaS companies preparing for an enterprise security review, the pipeline is the natural place to start, because that is where evidence and protection come from at the same time. For manufacturers of products covered by the Cyber Resilience Act, those records also support compliance work.
Tell us how your team ships today, and we will show you where the first checks belong.
Key takeaways
- AI-generated code is improving but not safe by default. Veracode’s October 2026 tests found vulnerable code in 38% of security-relevant tasks.
- Verification is the scarce skill. Only 26.0% of AI-generated patches in 1Password’s study were clean fixes, and SecureFlag found a quarter of developer fixes flawed, so keep a qualified human accountable for what ships.
- The pipeline is the control point. Secret scanning, dependency scanning and SAST on every change, DAST on every release, and AI assistant configuration reviewed like code.
- Deleting a secret is not revoking it. The LiteLLM incident began with a key removed from a repository but never rotated, and escalated through an owner-level token held by an AI agent platform.
- Know what you ship. An SBOM on every build is what makes it practical to meet obligations such as the Cyber Resilience Act’s 24-hour early warning for actively exploited vulnerabilities.
- Test continuously, close quickly. Continuous, AI-assisted testing paired with fast, verified remediation keeps an actively shipped product secure.
Frequently asked questions
Not by default. In Veracode's October 2026 tests, AI models produced code with a known vulnerability in 38% of security-relevant tasks, down from 44% in July, and even the best models failed more than one task in four. AI-generated code can be safe to ship once it passes the same scanning, review and testing you apply to any third-party contribution. The risk comes from merging it on trust.
Treat it as untrusted input. Run static analysis, dependency and secret scanning on every pull request, require human review on authentication, access control, payments and data handling, and verify that any package the AI suggests actually exists and is maintained. Keep coding assistants away from production credentials and real user data, and test the running application continuously rather than once a year. We cover the full sequence in this guide and in our DevOps consulting work.
Security by design means a product is conceived with customer security as a core business goal before development starts, rather than added as a feature later. CISA's joint guidance pairs it with secure by default, where products are safe out of the box with little configuration and no extra charge for security features. In engineering terms, it means threat modeling, safe defaults, eliminating whole vulnerability classes and keeping security checks inside delivery.
Know what you ship and control how it gets in. Generate an SBOM on every build, pin dependency versions with lockfiles, scan components for known vulnerabilities and malicious packages, and delay adoption of brand-new releases. Protect the pipeline itself: scoped, short-lived tokens, protected branches, reviewed workflow changes and signed releases. Assume a compromised package means a compromised host and rotate every credential it could reach.
Add checks in layers, starting with the cheapest. Secret scanning and dependency scanning run on every commit, static analysis runs on every pull request, and dynamic testing runs against staging on every release. Start by reporting findings, then block merges on new critical issues once the noise is tuned. These checks can usually be added to the pipeline and repositories you already use, without changing your stack.



