Choose the workflow before the tool
Start with a repeated task that someone can explain from beginning to end. Perhaps enquiries arrive in several inboxes, meeting notes need organizing or staff repeatedly search the same internal documents. Write down the inputs, decisions, output and person responsible. If the process itself is unclear, automation will be difficult to evaluate.
Ask whether a simpler rule or integration would solve the problem. Moving a form submission into a shared queue may only require ordinary automation. AI becomes an option when the task involves interpreting varied language or preparing a draft, and when the organization can review the result appropriately.
Define what the system may do
Give a pilot a narrow job. For an enquiry workflow, the system might suggest a category and prepare an internal summary. A person still decides how to respond. Separate a suggestion from an action, especially when an action sends information externally or changes a customer record.
- List the information the workflow is allowed to access.
- Identify outputs that always need human review.
- Define what happens when information is incomplete or contradictory.
- Keep a manual path available if the integration is unavailable.
Use test examples that resemble the real task without exposing unnecessary confidential information. Include unusual requests, missing fields and ambiguous wording. A useful pilot makes its limits visible instead of hiding them behind a polished demonstration.
Evaluate the entire handoff
A generated answer is only one part of the workflow. Check whether the information reaches the right person, whether they can trace the source and whether correcting an error is straightforward. If staff must spend longer verifying the output than doing the original task, the design needs another pass.
Before the pilot, choose a small set of evaluation criteria: successful routing, completeness, correction effort and handling of unsupported requests. Compare the assisted workflow with the existing one using the same examples. Keep a record of failures as well as successes so that improvements target the actual weakness.
Connect an assistant to the right knowledge
Consider an employee learning assistant or a product-information assistant as two different examples. The first needs to respect a person’s role and access permissions. The second needs to distinguish product facts from suggestions and keep its answers aligned with the available catalogue. Both begin with information quality and ownership, not a chat interface.
Retrieval-augmented generation, or RAG, supplies relevant material to a model before it prepares a response. Semantic search can help retrieve passages related to a question even when the wording differs. That does not make every retrieved passage correct, current or authorised. Keep document versions, access checks and source references in the application design, and test whether the retrieved material actually supports the answer.
- Identify the approved knowledge sources and their owners.
- Apply the user’s permissions before returning information.
- Make source references available for review.
- Define a response for missing, conflicting or outdated evidence.
- Evaluate retrieval and answer quality separately.
Separate an answer from permission to act
Tool calling connects an assistant to application functions. A proposed action might look up a product, prepare a lead summary or request a workflow step. The application still needs to validate inputs, enforce permissions and decide whether approval is required. A model-generated request is not evidence that the person is authorised to perform it.
Start with read-only or reversible tasks where the handoff is easy to inspect. Require human review for consequential actions and preserve a useful manual fallback. Test unavailable services, incomplete data and duplicate requests as carefully as the normal path. A convincing answer in a demo does not establish that a business integration will behave reliably.
Expand only after the narrow version works
Assign an operational owner, document access permissions and decide how changes will be tested. A workflow can drift when forms change, services are renamed or internal documents become outdated. Build review into normal operations rather than treating launch as the end of the project.
If the pilot cannot meet the agreed standard, stop or reduce its scope. The goal is a more dependable process. Sometimes that means AI assistance; sometimes it means better information, a simpler form or an ordinary integration.