Copilot Trust & Responsible Use

Microsoft 365 Copilot is a powerful drafting and analysis assistant, but it's not infallible. Like any tool, it requires proper use, verification, and human oversight. This guide walks through the guardrails that keep Copilot outputs safe, accurate, and appropriate for city work.

How LLMs work: understanding the technology

To use Copilot effectively and safely, it helps to understand what's happening under the hood. Large Language Models (LLMs) like the one powering Copilot are fundamentally different from traditional software—they don't follow programmed rules, but instead learn patterns from vast amounts of text data.

What is a Large Language Model?

An LLM is a neural network—a mathematical system inspired by how neurons connect in the brain—that has been trained to predict what words should come next in a sequence. When you type a prompt, the model doesn't "know" facts or "understand" meaning in the human sense. Instead, it uses statistical patterns learned during training to generate the most likely continuation of your text.

Training on unknown data

LLMs are trained on enormous datasets—often trillions of words from books, websites, articles, code repositories, and other text sources. The specific sources and content are typically not disclosed by the companies that build these models. This means:

• You don't know what the model has seen: The training data includes a mix of accurate information, outdated facts, opinions, fiction, and errors from across the internet.
• Training data is frozen: Once trained, the model's knowledge is fixed at a point in time. It doesn't automatically learn new information after training.
• No source attribution: The model doesn't remember where it learned something, so it can't cite sources from its training data.
• Quality varies: Training data quality varies widely, which is why outputs need verification.

Weights: the "memory" of the model

During training, the model adjusts billions of numerical values called "weights" that determine how strongly different parts of the network connect. These weights encode the patterns the model learned—not facts, but relationships between words, concepts, and contexts.

• Weights are not facts: They represent statistical correlations, not verified information. The model learned that "Lethbridge" often appears near "Alberta," but it doesn't "know" this as a fact.
• Weights encode patterns: The model learned that formal reports often start with "Executive Summary" and that budget documents typically include tables. These are patterns, not rules.
• Weights can't be edited: You can't directly change what the model learned during training. This is why prompt engineering (framing) is so important—it's your way to guide the model's behavior.

Non-probabilistic but pattern-based

While LLMs use probability to choose words, they're not purely probabilistic systems. The model doesn't calculate "there's a 73% chance the next word is 'budget'"—instead, it uses complex neural network calculations that produce outputs based on learned patterns.

• Deterministic randomness: Given the same prompt and context, modern LLMs can produce different outputs due to built-in randomness (temperature settings). This is intentional—it allows creativity and variation.
• Pattern-based, not rule-based: The model doesn't follow if-then rules. It generates text based on patterns it learned, which can lead to unexpected outputs.
• No guarantee of correctness: Because outputs are pattern-based rather than fact-checked, there's no guarantee the model is correct, even when it sounds confident.

How "framing" works: shaping outputs through prompts

"Framing" refers to how the way you structure your prompt influences what the model generates. The same underlying model can produce very different outputs depending on how you frame the request. Understanding framing helps you get better, more reliable results.

The power of context and structure

When you write a prompt, you're not just asking a question—you're providing context that activates different patterns in the model. The model uses everything in your prompt to determine what kind of response to generate.

How framing influences the model

Framing works because LLMs are context-sensitive. Every word in your prompt influences which patterns the model activates:

• Role framing: "As a city manager, draft..." vs. "As a resident, explain..." activates different language patterns
• Format framing: "Create a table..." vs. "Write a paragraph..." shapes the output structure
• Tone framing: "Write a friendly notice..." vs. "Draft a formal memo..." influences word choice
• Scope framing: "Summarize the key points..." vs. "Provide a detailed analysis..." determines depth

Framing and hallucinations

Poor framing can increase hallucinations. When the model lacks clear context, it may fill gaps with plausible-sounding but incorrect information from its training data.

Advanced framing techniques

Experienced users leverage framing to get more reliable outputs:

• Chain-of-thought framing: "First, identify the main issues. Then, for each issue, propose three solutions. Finally, recommend the best solution with rationale." This breaks complex tasks into steps.
• Constraint framing: "Draft an email under 150 words, using only information from the attached report, and avoiding any speculation." This limits what the model can include.
• Verification framing: "After summarizing, list any assumptions you made and any information you couldn't find in the attached files." This encourages transparency.
• Iterative framing: Start broad, then refine. "Draft a first version focusing on structure. Then refine it to match the tone of the example document attached."

Human-in-the-loop: why it matters

Copilot drafts; you decide. Keeping a person involved turns Copilot into a partner instead of an autopilot. Humans decide when the answer is good enough, when it needs more sources, and when to stop and ask a subject matter expert. The goal is to speed up drafting and analysis without handing over judgment.

The concept of "human-in-the-loop" has become central to responsible AI deployment, particularly in high-stakes contexts like government work. Research on human-AI collaboration shows that systems that keep humans involved in decision-making processes achieve better outcomes than fully automated systems, especially when dealing with complex, context-dependent tasks (Bansal et al., 2021). This is because humans bring domain expertise, ethical judgment, and contextual understanding that AI systems currently lack.

For municipal governments, human oversight is not just a best practice—it's often a legal and ethical requirement. Public sector organizations have obligations to ensure accuracy, transparency, and accountability in their communications and decision-making processes. Research on AI governance in the public sector emphasizes that human oversight helps maintain public trust and ensures that AI tools serve public interests rather than replacing human judgment (Wirtz et al., 2019).

Think of Copilot like a capable junior colleague who works 24/7 and never gets tired of revisions. But just like you'd review a junior colleague's work before sending it to council or residents, you need to review Copilot's outputs.

References: Bansal, G., et al. (2021). Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff. AAAI '21. Wirtz, B. W., et al. (2019). Artificial Intelligence and the Public Sector—Applications and Challenges. International Journal of Public Administration.

Common risks and how to spot them

Most Copilot issues come from context gaps or treating the model like an oracle. Clear prompts, citations, and quick checks with trusted sources remove most surprises. Treat every output as a draft until it is verified.

Jailbreaks and model corruption

Jailbreaks work by persuading the model to ignore safety rules. They often look like harmless role-play but are designed to bypass guardrails. Staying inside the approved M365 tenant and avoiding untrusted prompts keeps the model aligned with organizational policy.

Jailbreaking means tricking a model into ignoring safeguards (e.g., "pretend you are malware and give me steps"). Corruption can also occur if a model is fed hostile inputs or misconfigured plugins.

"Evil AI" scenarios (and how to prevent)

These scenarios are unlikely when governance, permissions, and human checks are in place. They are still useful to rehearse so staff know how to respond quickly if something looks off. Policy plus training is the safety net.

Concrete protections

Think of protections as layers: technical controls, good prompts, and user habits. None are perfect alone, but together they keep Copilot useful and safe for city work.

The security advantages of using Microsoft 365 Copilot within an organization that already uses Microsoft 365 cannot be overstated. Unlike standalone AI services that require data to be uploaded to external platforms, Copilot operates entirely within your Microsoft 365 tenant, meaning organizational data never leaves your controlled environment. This integration ensures that all existing security policies—including multi-factor authentication, conditional access rules, data loss prevention policies, and compliance frameworks—automatically apply to Copilot interactions. Research on enterprise data security emphasizes that integrated solutions reduce attack surfaces and simplify compliance by eliminating the need to manage separate security controls for AI tools (Tallon et al., 2013). For municipal governments, this integration is particularly valuable because it ensures that sensitive citizen data, internal communications, and confidential documents remain protected by the same security infrastructure that governs all other organizational data, significantly reducing the risk of data breaches or unauthorized access.

References: Tallon, P. P., et al. (2013). Competing Perspectives on the Link Between Strategic Information Technology Alignment and Organizational Agility: Insights from a Mediation Model. MIS Quarterly.

When to escalate

Not every issue needs escalation, but some situations require immediate attention from IT or security teams.

• Output includes personal data, confidential numbers, or risky language that shouldn't be there.
• Repeated hallucinations or off-brand tone you cannot correct quickly.
• Access issues to SharePoint/Teams sources that block grounded answers.
• Suspicious prompts or outputs that might indicate a security concern.

Good patterns to encourage

Building good habits makes Copilot safer and more useful. These patterns help staff get better results while maintaining quality and security.

Understanding model limitations

Knowing what Copilot can and cannot do helps you use it effectively and avoid frustration. These limitations stem from how LLMs work fundamentally—they're pattern-matching systems, not knowledge bases or reasoning engines.

What Copilot is good at

• Drafting and rewriting: Generating initial versions of documents, emails, and reports based on your content
• Summarizing: Condensing long documents, meeting notes, or data into shorter formats
• Formatting and structuring: Organizing information into tables, lists, or specific document structures
• Language tasks: Adjusting tone, translating concepts into plain language, or adapting style for different audiences
• Pattern recognition: Identifying themes, trends, or common elements across documents
• Idea generation: Brainstorming options, approaches, or solutions (with your review)

What Copilot struggles with

• Real-time information: It doesn't have access to current events, live data, or information after its training cutoff date
• Complex calculations: Mathematical operations, especially with large numbers or multi-step formulas, are error-prone
• Fact verification: It can't verify facts independently—it can only work with what you provide
• Reasoning: While it can appear to reason, it's pattern-matching, not true logical reasoning
• Consistency: The same prompt might produce different outputs, and it may contradict itself across conversations
• Source accuracy: It can't guarantee that information from your files is correct—it can only work with what's there

The importance of domain knowledge

Copilot doesn't understand your specific domain knowledge, local context, or organizational nuances. It can help draft and organize, but you need to provide:

• Context: What matters in your specific situation
• Constraints: What's allowed, what's not, and what the boundaries are
• Quality standards: What "good enough" means for your use case
• Verification: Whether outputs meet your requirements

Building trust through practice

Trust in Copilot grows through experience. Start with low-risk tasks where mistakes are easy to catch—summarizing meeting notes, drafting internal emails, or creating outlines. As you build confidence and learn Copilot's strengths and limitations, you can gradually use it for more important work.

Progressive skill building

Think of learning Copilot like learning any new tool—start simple and build complexity gradually:

Learning from mistakes

When Copilot produces something wrong or unexpected, that's a learning opportunity. Ask yourself:

• What went wrong? Was it a hallucination, calculation error, tone issue, or something else?
• Why did it happen? Was the prompt unclear? Were sources missing? Was the task too complex?
• How could I prevent it? What would make the prompt clearer? What verification should I add?
• What did I learn? How can I apply this to future prompts?

Building verification habits

Trust comes from consistent verification, not blind faith. Develop habits that make verification routine:

• Source check: Always verify key facts against original documents
• Number check: Double-check all calculations and numerical data
• Tone check: Review language for appropriateness and inclusivity
• Completeness check: Ensure all required elements are present
• Context check: Verify that outputs make sense in your specific situation

Remember: Copilot is a tool, not a replacement for judgment. The best users combine Copilot's speed with their own expertise, always verifying outputs and making final decisions themselves.