Author name: G360 Technologies

BitBypass: Binary Word Substitution Defeats Multiple Guard Systems BitBypass changes model behavior by hiding a single sensitive word as binary bits. The method requires no model weights, no gradients, and no complex adversarial optimization. It works by encoding one keyword as a hyphen-separated bitstream and instructing the model to decode it. In testing across five frontier models, this technique dropped refusal rates from ranges of 66 99% down to 0 28% and induced all five models to generate phishing content at rates between 68 92%. The BitBypass paper (“BitBypass: A New Direction in Jailbreaking Aligned Large Language Models with Bitstream Camouflage” was posted to arXiv on June 3, 2025 (arXiv:2506.02479) and accepted to EACL 2026. Findings based on a Texas A&M SPIES Lab post dated January 5, 2026. The authors evaluate BitBypass against five LLMs: GPT4o, Gemini 1.5 Pro, Claude 3.5 Sonnet, Llama 3.1 70B, and Mixtral 8 22B. They test bypass behavior against multiple guard systems: OpenAI Moderation, Llama Guard (original), Llama Guard 2, Llama Guard 3, and ShieldGemma. The evaluation uses standard harmful-instruction benchmarks, AdvBench and Behaviors) plus a phishing-focused benchmark introduced by the authors, PhishyContent, consisting of 400 prompts across 20 phishing categories hosted on Hugging Face. The evaluation includes a refusal judge and an LLM-based judge for harmfulness and quality, with phishing-specific classification handled by a dedicated harm judge. How the mechanism works BitBypass operates under an “Open Access Jailbreak Attack” threat model. The attacker has API access to a commercial LLM and control over inference-time parameters, including the system prompt, user prompt, and decoding settings. The attacker does not need model weights, gradients, or training data. The core idea is to hide a single sensitive word in a harmful instruction by encoding it as bits while keeping the rest of the instruction in natural language. 1. Bitstream camouflage in the user prompt The attacker selects one sensitive keyword in an otherwise harmful request. That word is converted into an ASCII binary representation and formatted as a hyphen-separated bitstream. In the natural-language instruction, the sensitive word is replaced with a placeholder token such as [BINARY_WORD] . The user message includes both the bitstream and the partially redacted instruction so the model has enough context to reconstruct the original request. The result is an input that looks like benign “data plus template text” to both humans and simple filters, because the sensitive token is no longer present in plain language. 2. A system prompt that forces decoding and reconstruction. Three system-prompt components drive the attack: Curbed Capabilities: System-level instructions that explicitly constrain or redirect default safety behavior and push the model to prioritize the decoding and task-following instructions. The ablation study shows this is the most critical component. Effectiveness drops sharply when removed. Program-of-Thought: The system prompt includes a Python-like function (named bin_2_text) and instructions that guide the model to conceptually decode the bitstream back into text. This is conceptual rather than executed in an actual interpreter. The function does not fully handle the hyphenation format, relying on the model’s reasoning to bridge that gap. Focus Shifting: After decoding and reconstructing the request internally, the prompt sequence shifts the model into subsequent steps or tasks. This reduces the chance that safety behavior triggers at the moment the reconstructed sensitive term becomes salient again. 3. Why guard models can miss it Guard models are independent filters that classify prompts for policy violations. BitBypass exploits a gap: the guard model sees bitstrings and a placeholder rather than the reconstructed sensitive word and completed harmful request. Some guard models show more resilience than others Llama Guard 2 and 3 , but meaningful bypass rates remain across all tested systems. Why this matters BitBypass works because it is simple and repeatable, not because it is sophisticated. It uses a deterministic encoding of a single word and relies on the model’s general ability to interpret structured representations when instructed. That simplicity is the problem. Direct harmful instructions trigger refusal. BitBypass substantially reduces refusal and increases unsafe output generation across multiple models. Testing shows a shift from high refusal rates (roughly 66 99% under direct instructions) toward much lower refusal rates under BitBypass 0 28% , with corresponding increases in attack success rates (roughly 48 78% for harmful-instruction benchmarks). The phishing results map directly to enterprise abuse patterns. Under BitBypass, all five tested models produced phishing content at high rates on the PhishyContent benchmark 68 92% phishing content rate across models). This is not a theoretical risk. Phishing infrastructure, credential harvesting, and business email compromise are operational threats that enterprises face daily. Implications for enterprises 1. System prompt control is now a first-order security control The BitBypass threat model assumes an attacker can influence the system prompt. Many enterprise deployments do not allow this directly, but agent frameworks, tool routers, multi-tenant “prompt templating,” and “bring your own system prompt” features can unintentionally widen that surface area. If untrusted users can shape or inject system instructions, BitBypass-style patterns become feasible. 2. Input screening that relies on natural-language semantics has structural limits BitBypass is an example of “non-natural language adversarialism,” where the disallowed intent is split between an encoded fragment and a decoding procedure. Controls that focus on keyword triggers, typical jailbreak phrases, or standard natural-language toxicity signals will underperform if they do not address structured encodings and transformation steps. 3. Guard models help, but their coverage varies Testing shows wide bypass-rate ranges for guard systems under BitBypass (roughly 22 93% depending on guard model and dataset), with Llama Guard 2 and 3 showing more robustness than some alternatives. For enterprise architecture, this means measured evaluation of the specific guard model in use, plus continuous testing against encoding-based attacks rather than assuming “a moderation layer” is sufficient. 4. Testing needs to include encoded and reconstruction-based abuse cases The evaluation uses AdvBench, Behaviors, and PhishyContent to point to a practical testing direction: jailbreak evaluation suites should include structured encodings, reconstruction steps, and mixed-format prompts, not only straightforward malicious instructions and roleplay-based jailbreaks. Risks and open

NIST’s Cyber AI Profile Draft: How CSF 2.0 Is Being Extended to AI Cybersecurity A security team is asked to “do a CSF assessment” for a new AI assistant that connects to internal content and external model APIs. Everyone agrees CSF is the right backbone, but the team keeps getting stuck on the same questions: What counts as an AI asset? Where do prompts, model access, and training data fit? How do you describe AI-specific threats without creating a parallel framework? NIST’s new draft profile is an attempt to make that mapping concrete. In December 2025, NIST published an Initial Preliminary Draft of the Cybersecurity Framework Profile for Artificial Intelligence NIST IR 8596 iprd), positioned as a CSF 2.0 Community Profile focused on AI-related cybersecurity risk. The draft ran a public comment period from December 16, 2025, through January 30, 2026. NIST also scheduled a follow-on workshop on January 14, 2026, to discuss the preliminary draft. The Cyber AI Profile is designed to integrate into existing cybersecurity programs rather than replace them. It is organized around the NIST Cybersecurity Framework CSF 2.0 and coordinated with other NIST risk frameworks that organizations already use. Two pieces of context matter for how to read the document: It is an “Initial Preliminary Draft.” NIST explicitly framed it as an early release to share current thinking and solicit feedback before an Initial Public Draft and a final profile. It intentionally avoids a narrow definition of “AI.” The draft uses “AI systems” broadly, covering stand-alone AI systems and AI embedded into other applications, infrastructure, and processes. NIST ties the profile into a larger set of NIST AI risk work, including the AI Risk Management Framework AI RMF 1.0, released January 26, 2023, and the Generative AI Profile NIST AI 600 1, published July 26, 2024. How The Mechanism Works At its core, the Cyber AI Profile is a structured overlay on CSF 2.0. It starts with CSF 2.0 outcomes. The profile is organized by the CSF 2.0 Functions and their Categories and Subcategories. In the draft, this is implemented as a set of tables aligned to each CSF Function: GOVERN, IDENTIFY, PROTECT, DETECT, RESPOND, and RECOVER. It adds three AI Focus Areas. For each CSF outcome, the profile layers AI cybersecurity considerations through three Focus Areas: Secure (cybersecurity of AI system components and the ecosystem they rely on), Defend (use of AI capabilities to improve cyber defense activities), and Thwart (resilience against adversaries using AI to enhance attacks). These Focus Areas are meant to structure AI-related cybersecurity risk without creating a separate framework taxonomy. It uses table columns to connect outcome to AI-specific guidance. For each CSF Subcategory, the draft provides general considerations (baseline cybersecurity considerations), focus-area-specific considerations that describe AI-relevant threats, mitigations, and implementation details under Secure, Defend, and Thwart, proposed priority signals for focus-area work (the draft uses a 1 3 scale to indicate where organizations may focus first), and example informative references, with NIST noting the list is incomplete and undergoing further literature review. It explicitly solicits feedback on usability and structure. The draft explicitly solicits feedback on how stakeholders would use the profile, whether Focus Areas should be presented together or separately, preferred delivery formats (including tooling-oriented formats), and what glossary terms and informative references should be added. What This Actually Forces Into The Open This draft matters because it takes a problem many enterprises already have and forces it into a consistent control language: how to treat AI systems as part of normal cybersecurity risk management while still acknowledging that AI introduces distinct attack surfaces and failure modes. The immediate consequence is visibility. Teams that have been running AI pilots without formal asset classification now have to answer: where is the model hosted, who can access it, what data does it touch, and what happens if it gets compromised or starts behaving unexpectedly? The profile does not allow those questions to stay vague. CSF mapping requires explicit answers, which means AI systems that were treated as “innovation projects” become governed infrastructure with incident response obligations. The structure is also a signal. By publishing this as a CSF 2.0 Community Profile, NIST is making a specific governance move: AI cybersecurity risk is expected to map to the same enterprise cybersecurity outcomes used for everything else, including governance, asset identification, protective controls, detection, response, and recovery. Organizations that built AI security programs in parallel to their existing cybersecurity frameworks now have a forcing function to consolidate. The timing is deliberate. The draft was published in December 2025, with an immediate comment window and a January 2026 workshop, indicating NIST is actively pulling industry input to refine both the content and the practical form factor before the next draft stage. The speed suggests NIST expects this to move quickly from draft to operational guidance. Implications for Enterprises Operational Implications Program integration work becomes clearer, but more explicit. Teams that already operate CSF-based assessments can use the profile to structure AI cybersecurity discussions in familiar CSF terms instead of inventing AI-only assessment categories. The trade-off is that AI systems can no longer be evaluated in isolation. If a marketing team deploys a chatbot that connects to a third-party API, that deployment now requires the same level of asset documentation, access control review, and incident response planning as any other system that handles enterprise data. Inventory and dependency mapping pressure increases. The profile’s CSF alignment pushes organizations toward an explicit view of AI systems and their dependencies as governed assets, including embedded AI, not only obvious stand-alone deployments. This is where the friction shows up. Teams have to identify not just the chatbot, but the API it calls, the authentication mechanism it uses, the data sources it accesses, and the logging infrastructure that captures its behavior. Many organizations do not have that level of visibility today, especially for AI integrations that were deployed quickly or embedded into existing tools. Incident response and recovery planning must include AI artifacts. The profile’s RESPOND and RECOVER alignment

Structured Outputs Are Becoming the Default Contract for LLM Integrations A team ships an LLM feature that returns JSON for downstream automation. In testing, it mostly works. In production, a small percentage of responses include an extra sentence, a missing field, or a value outside an enum. Each case becomes a validation failure, retry, or brittle parsing code that quietly enters the system’s reliability budget. For the past two years, production LLM integrations have relied on a fragile contract: ask the model politely to return JSON, then write defensive parsing code for when it doesn’t. That pattern is being replaced. Across provider APIs and open-source inference stacks, structured outputs are becoming first-class infrastructure, with schema enforcement moved into the decoding layer rather than application code. OpenAI moved from JSON mode, which guarantees valid JSON but not schema adherence, to Structured Outputs that enforce a supplied JSON Schema when strict mode is enabled. In parallel, vLLM and adjacent tooling have made structured outputs a core serving feature, with explicit migration away from older guided parameters toward a unified structured outputs interface. The old pattern looked reasonable in demos. Prompt the model to output JSON, parse the response, validate against a schema, retry on failure. JSON mode reduced syntax breakage but left schema drift, missing required keys, and invalid values as application problems. Every production system that depended on reliable structured data ended up with the same stack of validation logic, retry loops, and error handling. OpenAI’s Structured Outputs reframes this as an API contract: when strict mode is used with a JSON Schema, the model output is constrained to match the schema. On the open-source serving side, vLLM treats structured outputs as a core capability with multiple constraint types and server-side enforcement. Maintainer discussions and redesign work in vLLM’s V1 engine are explicitly motivated by performance and throughput concerns when structured output requests are introduced at scale. How the mechanism works Structured output enforcement is implemented as constrained decoding. Instead of letting the model sample any next token from its full vocabulary, the decoder restricts the set of allowable next tokens so that the growing output remains consistent with a formal constraint such as a JSON Schema, regex, or grammar. Implementations commonly compile the constraint into a state machine or grammar matcher that can decide, at each step, which tokens would keep the output valid. The decoding loop applies those constraints while generating tokens. vLLM’s documentation and engineering writeups describe this as structured output support with backends such as xgrammar or guidance-based approaches. At the library layer, projects such as llguidance describe constrained decoding as enforcing context-free grammars efficiently, and Outlines positions itself as guaranteeing structured outputs during generation across multiple model backends. The technical shift is straightforward: move the validation problem from your application into the inference engine. Analysis This matters now because structured outputs are moving from nice-to-have prompt hygiene into contract-level infrastructure that toolchains are standardizing around. OpenAI’s Structured Outputs make schema conformance an explicit API-level behavior in strict mode, which removes the operational burden of validation and retry loops for schema shape issues. In inference stacks, vLLM’s V1 engine work treats structured outputs as a feature that must not degrade system throughput, and maintainers explicitly call out performance as a blocker to feature parity. Constrained decoding is being measured and benchmarked as a standard production technique. A 2025 evaluation paper on structured generation reports that constrained decoding can improve generation efficiency relative to unconstrained decoding while guaranteeing constraint compliance. The API surface is converging. vLLM now warns about deprecated fields and directs users to use a unified structured_outputs interface. Server-side protocol definitions mark older guided knobs as deprecated with planned removal timelines. The ecosystem is settling on a shared approach. Implications for enterprises Operational implications Fewer format incidents, more content incidents. When schema shape errors drop, the remaining failures are semantic: incorrect extracted values that still fit the schema. Structured outputs improve reliability of form, not correctness of meaning. This shifts QA effort toward evaluation of content quality and downstream controls rather than parsing resilience. The failure modes change, not the failure rate. Platform standardization pressure. As provider APIs and inference stacks converge on schema-driven interfaces, platform teams will face pressure to offer a standard contract mechanism across internal products rather than letting each team invent its own parsing and retry logic. The pattern is becoming infrastructure, which means it needs infrastructure-level support. Migration work is real work. Deprecations and interface changes become part of platform lifecycle management, with version pinning, integration testing, and rollout planning. Teams that built on older guided parameters now have migration paths to follow and timelines to track. Technical implications Schema design becomes an integration surface. If the schema is the contract, it needs the same discipline applied to internal APIs: explicit compatibility expectations, careful changes, and documented consumer assumptions. OpenAI’s strict schema enforcement and vLLM’s structured outputs both make the schema a first-class input to the generation pipeline. A breaking schema change is a breaking API change. Backend behavior and failure modes matter. vLLM issue discussions document cases where the structured output finite state machine can fail to advance in the xgrammar backend, and the engine may abort the request in response. That is a production failure mode enterprises need to monitor, alert on, and handle with fallbacks where appropriate. The guarantee is stronger, but the failure is harder. Performance is part of the contract. vLLM’s structured outputs work, and RFCs explicitly treat performance challenges as a blocker to feature parity. Constrained decoding is not free, even if it is trending toward minimal overhead in mature implementations. Teams need to measure throughput impact when enabling structured outputs at scale. Risks and open questions Schema compliance can hide semantic failure. A perfectly valid JSON object can still contain incorrect or low-quality values. Structured outputs reduce certain classes of brittleness but do not guarantee the correctness of the underlying facts or extraction decisions. The risk is that teams treat schema conformance as

When Prompts Started Breaking Production A team updates a system prompt to reduce hallucinations. The assistant sounds better in demos, but a downstream parser starts failing because formatting shifted in subtle ways. Nothing in the application code changed, so traditional tests stay green. The only signal is a rising error rate and escalations. This is the operational shape of prompt regressions: the system is up, but behavior is outside contract. By early 2026, prompts were breaking production systems often enough that engineering teams stopped treating them as configuration and started treating them like code. The pattern: version prompts, define regression suites, run automated evals in CI/CD, and block deployments when metrics fall below gates. This is test-driven prompt engineering. In early prompt workflows, iteration looked like trial-and-error in a playground, validated by a handful of manual examples. By 2025, that approach had produced enough incidents that multiple sources described the same shift: prompt test suites and evaluation loops that resemble software QA and release engineering. Several strands converged. “Test-Driven Prompt Engineering” writeups framed prompts and evals as code and tests, with explicit versioning and regression practices. Platform tooling emphasized dataset-based evaluation runs triggered by prompt changes in CI systems. Product teams documented evaluation-driven refinement on real assistants. And incident narratives kept highlighting the same failure mode: prompt modifications, unauthorized or accidental, created safety failures, format breakage, or drift that traditional QA never caught. In parallel, evaluation extended beyond single-turn correctness to agent behavior, including tool use and multi-step workflows. The bar for what “tested” means in LLM systems went up. How the mechanism works Evaluation-driven prompt engineering is a lifecycle that treats prompts as managed release assets with measurable acceptance criteria. Five practices define it: 1. Versioned artifacts Instead of embedding prompts as string literals, teams store them as distinct files or registry entries and version them, often with semantic versioning. Some workflows pin prompts to specific model snapshots to avoid surprises from provider alias updates. The practical effect is traceability: teams can answer which prompt version produced a given output and roll back quickly. 2. Test suites and datasets A prompt test suite is a structured set of test cases that represent expected behavior. Test cases may include explicit expected outputs, but often they include evaluation criteria: format constraints, required elements, tool-call correctness, tone requirements, or groundedness against provided context. Golden datasets are curated from core workflows and failure cases. Some systems enrich them with security probes or scenario generation to expand coverage. Research on multi-prompt evaluation argues that single-prompt testing misses variance caused by small wording differences, which supports using suites that evaluate multiple prompt variants per case. 3. Scoring models Common checks include format and schema compliance, for example, JSON parseability or contract adherence, plus keyword, regex, or structural checks for required elements. Task success scoring, sometimes as a percentage of cases that meet criteria. Hallucination or faithfulness scoring, often using an LLM-asjudge approach against the provided context. Safety and policy checks, including redteam style probes for jailbreak and prompt injection patterns. Operational metrics like latency distributions and token cost per case. Because LLM behavior is nondeterministic, many workflows use pass rates, thresholds, and slice-based evaluation rather than single binary assertions. 4. CI/CD gates When prompt files or templates change, CI triggers the evaluation suite. If key metrics regress beyond thresholds, the pipeline fails and the change is blocked from deployment. Some playbooks include post-deploy monitoring and automated rollback if production metrics fall below guardrails. 5. Production feedback Several sources describe monitoring prompt quality alongside traditional SRE metrics. The insight is that prompt-related failures can be silent: the service is healthy by uptime metrics while semantic quality degrades. Teams address this by tracking quality metrics over time and feeding new failure cases back into the evaluation dataset. Analysis This pattern emerged because prompts are no longer a side input to a model. In many enterprise systems, prompts define behavior, policy constraints, and output contracts. When that interface changes, you can get outages, compliance issues, or workflow breakage without a code diff that triggers standard QA. Late-2025 incident narratives sharpened the problem from multiple angles. In May 2025, an unauthorized prompt change at xAI’s Grok service created a safety failure that made headlines. LinkedIn posts from November and December 2025 documented system prompt QA gaps and a Gemma hallucination incident where model behavior drifted without any prompt change at all. These are representative examples, not isolated cases. They clarified the risk: unauthorized or poorly controlled prompt changes can create safety and policy failures, turning prompt governance into a change-management problem. Model and tool behavior can drift, producing regressions without prompt changes. This motivates continuous regression testing and parallel evaluation across versions. Multi-provider failover improves availability but increases evaluation workload because prompts must be validated across the fallback chain, not just the primary provider. And prompt changes intended to improve one dimension, like hallucination reduction, can degrade another dimension, like format stability. Without contract-aware tests, downstream systems take the hit. The consistent theme is operational accountability. If prompts can trigger production incidents, they need the same discipline as other production configuration. Implications for enterprises Operational implications Release management: Prompt changes need an approval and promotion workflow, with versioning, diffing, and rollback. This includes system prompts, not just user-visible templates, since system prompt drift can bypass traditional QA. Incident response: Prompt versions must be observable during incidents so teams can correlate behavioral changes to a specific prompt or model update and roll back fast. The teams that caught regressions quickly in 2025 had prompt versioning already in place. The teams that struggled were still hunting through code commits to find what changed. Vendor resilience: If you implement provider failover, your eval footprint increases because you now need confidence in behavior across multiple model families and configurations. One source described this as the hidden cost of resilience: you pay for availability in evaluation work. Quality budgeting: Teams should plan for evaluation as a recurring operational cost, not a one-time integration