How to choose an AI tool for aviation compliance

Choose the tool that can show you, for every answer, the paragraph and the revision it came from in documents your organisation controls, that tells you when those documents are silent, that keeps your data inside the boundary your Part-IS risk assessment drew, and that leaves every decision with a named person. Features, speed and the size of the model matter less than those four, because they decide whether an answer can be checked in seconds or has to be done again by hand. This guide sets out the four kinds of tool that answer the same search, ten checks to apply to any of them, a 30-minute test you can run with questions whose answers you already know, and how Wingman360 Teammate answers each check, including the one where a buyer should press us.

Four kinds of tool answer the same search

"AI for aviation compliance" covers tools built for different jobs, and the comparison only makes sense once they are separated. They are not rivals so much as layers, and an organisation can end up running more than one.

Ten checks to apply to any tool

The first five come from the four ways a general model fails on regulatory questions, described in Can AI be trusted for aviation compliance documentation?: invented references, the wrong revision, a blurred hierarchy between rule and guidance, and mixed jurisdictions. The rest come from what the regulations ask of the organisation that uses the tool.

1. The citation opens at the paragraph. Every statement in an answer should point to a paragraph you can open in one step, not to a document or a web page. A citation you cannot open is a wrong answer, however plausible the text around it.
2. You know which revision it answered from. An answer that was right two amendments ago is a finding today. Ask the tool, or its vendor, which issue of the Easy Access Rules or of your exposition the answer came from, and who decides when that changes.
3. It says so when the source is silent. Ask something your documents do not cover. A tool fit for compliance work says it cannot find it; a tool built for general use answers anyway, from whatever it learned elsewhere.
4. It keeps rule, AMC and GM apart. An implementing rule is binding, an acceptable means of compliance is one way to comply, and guidance material explains. An answer that gives all three the same "shall" will put the wrong obligation into your procedure.
5. It does not mix jurisdictions. EASA Part 145 and the FAA's Part 145 differ in structure and in the forms they reference, and CS-23 is not 14 CFR Part 23. Ask a question in one jurisdiction's terms and check that the answer stays there.
6. It answers from your own documents. The public rules tell you what is required. Your exposition, handbook and procedures tell you how your organisation does it, and those are the documents staff need answers from. A tool that only knows the public rulebook answers half of the question.
7. Only named people can change what it knows. If any user can upload a document, forty engineers produce forty sources at forty revisions. Look for a controlled ingestion step with a named person accepting each document, and read-only use for everyone else.
8. Your data stays inside the boundary you assessed. Find out where the tool runs, which interfaces it creates, which model calls leave your network, and whether anything you enter trains a model. Those are the facts your Part-IS risk assessment needs.
9. Every output is a draft with a named reviewer. No tool holds an approval to issue compliance documents. The tool proposes, and a nominated person reviews and signs. A vendor who claims otherwise is claiming something no authority grants.
10. The quality claim comes with a method. A score without the task, the scoring rule and the comparison behind it is a slogan. Ask for a published measurement you could repeat on your own documents.

The obligation behind checks two and six is written into the rules. Point 145.A.45 makes an EASA maintenance organisation responsible for holding and using applicable current maintenance data and for keeping the data it controls up to date, and 14 CFR 145.109(d) requires an FAA certificated repair station to keep the documents and data for its work current and accessible when the work is being done. The obligation behind check eight is Part-IS: point IS.I.OR.205 asks the organisation to identify its systems, data and the services it receives, and the interfaces with other organisations that could expose it to information security risks, and to assess the risks with a potential impact on aviation safety; point IS.I.OR.235 applies if any part of its information security management activities is contracted to another organisation. The deployment questions to put to a vendor under those points are in the on-premise and air-gapped guide.

What the regulators have and have not said

EASA published its first AI rulemaking proposal, NPA 2025-07, on 10 November 2025. It proposes trustworthiness specifications for AI-based assistance and human-AI teaming, covers data-driven AI-based systems, and in EASA's words "will be extended in the future to reinforcement learning, knowledge-based technologies, hybrid and generative AI systems". It is the first step of rulemaking task RMT.0742, and a second NPA in 2026 is to apply the framework to the rules of each aviation domain. None of it approves a tool you can buy today. Checks one to ten are how you meet the obligations you already have while those rules are written.

A 30-minute test to run on any tool

Vendor demonstrations use questions the vendor chose. This test uses yours, and it needs nothing but the tool, your own documents and half an hour.

1. Write five questions you can already answer. Take them from your own exposition, handbook or procedures and from the rules behind them, and write down the answer and the paragraph for each before you start. You are testing the tool, so you need to know the right answer first.
2. Make one question about a recent amendment. Pick a point that was amended in the last two years and note the current wording. A tool answering from an older revision will give you the old answer with full confidence, and this is where you see it.
3. Make one question your documents do not answer. Choose something outside your corpus but inside aviation, so the tool has every temptation to answer from general knowledge. The right result is a clear statement that the source does not contain it.
4. Make one question where the rule and the AMC differ. Pick a point where the implementing rule sets the obligation and the acceptable means of compliance describes one way to meet it. Check that the answer tells you which is which and does not turn the AMC into a requirement.
5. Make one question in the other jurisdiction's terms. If you work to EASA rules, phrase one question in FAA wording, or the reverse. Check that the answer either maps the terms correctly or tells you the question belongs to the other regulator.
6. Run the five and open every citation. Put the same five questions to each tool you are comparing, in a fresh session, without extra prompting. For every answer, open each citation and read the paragraph it points to, at the revision the tool claims to have used.
7. Score what you can check, not what reads well. An answer passes only if every citation opens at a paragraph that says what the answer says, at the current revision. A clear refusal on the third question is a pass. One invented citation fails the tool, not only the question, because you cannot tell in daily use which answers carry one.

The same idea, run at scale with a scoring rule, is the RDB-1 Regulated Document Benchmark, which scored the technical report each tool produced against its sources. The general agent tool scored 66 out of 100; the best local model on Wingman360 Teammate scored 74, level with the best cloud model on the same harness.

How Wingman360 Teammate answers the ten checks

It is not a general assistant and does not try to be one: it will not write your supplier email or debug a script, and the comparison with ChatGPT, Claude and Gemini sets out where each belongs. What data it holds, where it runs and what it will not do are answered one question at a time on the trust page, and the segment pages show what it answers from in a design organisation, a UAS operator or a maintenance organisation.