The AI automation space is crowded with agencies that sprung up last Tuesday, learned n8n over the weekend, and are now charging $5,000 to connect your CRM to your email. Some are competent. Many are not. A few are outright dangerous — building brittle systems that break when you're not looking, locking you into platforms you can't escape, and vanishing when something goes wrong.
This guide is the vetting framework we wish existed when we were hiring agencies before building our own practice. It covers 7 red flags, the specific questions to ask, what good and bad answers sound like, and a complete document checklist.
Use this before you sign anything. It might save you $20,000 and six months of frustration.
Red Flag #1: They Can't Explain Your Workflow Back to You
Why it matters: An agency that doesn't deeply understand your business will build the wrong thing. Technical skill is useless without business context.
Specific questions to ask:
- "Walk me through my lead-to-close process as you understand it."
- "What are the 3 most important metrics this automation should improve?"
- "Where does this workflow break today, and why?"
What a good answer sounds like: They describe your process accurately, ask clarifying questions about edge cases, and identify failure points you hadn't considered. They reference similar workflows they've built and explain what they learned.
What a bad answer sounds like: Generic jargon ("We'll optimize your funnel with AI-driven touchpoints"), no follow-up questions, or a rush to talk about their "proven methodology" instead of your actual business.
Document checklist: Discovery call notes showing they understood your workflow. Written scope document with your specific process mapped. Definition of success tied to your metrics, not their deliverables.
Red Flag #2: No Clear IP or Asset Ownership Terms
Why it matters: If you don't own the code, configurations, and documentation, you're renting your own automation. When the engagement ends — or when they raise prices — you're stuck rebuilding from scratch.
Specific questions to ask:
- "Who owns the workflows, code, and configurations after delivery?"
- "Can I export everything and run it without you?"
- "What happens to my automations if we stop working together?"
- "Do you use proprietary tools I can't access?"
What a good answer sounds like: "You own everything. We'll deliver source code, configuration files, documentation, and video walkthroughs. You can host it yourself, hire someone else, or come back to us. No proprietary lock-in."
What a bad answer sounds like: "We retain ownership of the IP but you get a license." / "It's built on our custom platform." / "You can export some things but the core logic stays with us."
Document checklist: Contract clause explicitly stating client ownership of all deliverables. List of all tools/platforms used (verify none are proprietary/locked). Export test: ask them to show you exporting a workflow before you sign.
Red Flag #3: Pricing That's Vague or Too Good to Be True
Why it matters: $1,500 for "full business automation" is either a scam or a template dump. Vague pricing ("it depends on complexity") without a framework for what drives cost means surprise invoices later.
Specific questions to ask:
- "What's your pricing model — fixed, hourly, or retainer?"
- "What's included and what's extra?"
- "Can you give me a range for a workflow like [describe yours]?"
- "What happens if scope changes mid-project?"
What a good answer sounds like: Clear pricing tiers or a transparent cost framework. Honest discussion about what increases cost. Written change order process.
What a bad answer sounds like: "Let's start and see where it goes." / "We're very affordable compared to competitors." / Refusal to give even a rough range.
Document checklist: Written estimate with line items. Scope boundaries clearly defined. Change order process in writing. Payment terms (avoid 100% upfront).
Red Flag #4: No Maintenance or Handoff Plan
Why it matters: Automation isn't "set and forget." APIs change. Tools update. Your business evolves. An agency that builds and vanishes leaves you with a ticking time bomb.
Specific questions to ask:
- "What happens when an API breaks or a tool updates?"
- "How do you hand off knowledge to my team?"
- "What documentation do you deliver?"
- "Do you offer support after launch, and what does it cost?"
What a good answer sounds like: A clear handoff process: documentation, video walkthroughs, a transition call, and optional support tiers. They acknowledge that things break and have a plan for it.
What a bad answer sounds like: "It'll just work." / "Documentation is the workflow itself." / No mention of handoff. / Support only available at expensive retainer rates with no other option.
Document checklist: Documentation requirements in contract. Handoff session scheduled before final payment. Support terms defined. Escrow or holdback for post-launch verification.
Red Flag #5: They Recommend the Same Stack for Everyone
Why it matters: Every business has different needs, team skills, and existing tools. An agency that pushes their "preferred platform" regardless of your situation is optimizing for their convenience, not your outcomes.
Specific questions to ask:
- "Why are you recommending this specific tool/stack?"
- "What alternatives did you consider for my use case?"
- "What happens if we outgrow this platform?"
- "Does anyone on my team need technical skills to maintain this?"
What a good answer sounds like: A reasoned recommendation based on your team's skills, budget, volume, and existing tools. Discussion of 2-3 alternatives and why they were rejected. Honest admission of tradeoffs.
What a bad answer sounds like: "We only build on [Platform X]." / "It's the best, everyone uses it." / No discussion of alternatives. / Dismissal of your existing tools ("We'll just replace everything").
Document checklist: Written tech recommendation with rationale. Migration plan from current tools. Exit plan — how to move away from recommended stack if needed.
Red Flag #6: No References or Portfolio Depth
Why it matters: Anyone can claim expertise. Proof is in production systems that have been running for months, with clients who'll actually talk to you.
Specific questions to ask:
- "Can I speak to a client who's had automation running for 6+ months?"
- "Show me a workflow similar to what you're proposing for me."
- "What's the longest-running automation you've built?"
- "Have any clients left? Why?"
What a good answer sounds like: They provide 2-3 references without hesitation. They show you live systems (not mockups). They acknowledge a client who left and explain why — usually a mismatch in expectations or scope, which is honest and normal.
What a bad answer sounds like: "We keep client information confidential." / Only case studies with no named clients. / References who are "too busy." / All portfolio items are less than 3 months old.
Document checklist: 2-3 reference calls scheduled. Screenshots or video of live workflows. Check how long the agency has existed (LinkedIn, domain registration).
Red Flag #7: They Promise AI Will "Transform Your Business"
Why it matters: AI is a tool, not magic. Agencies that sell transformation instead of specific workflow improvements are either naive or dishonest. Real automation delivers measurable, narrow gains — hours saved, errors reduced, speed increased.
Specific questions to ask:
- "What specific metric will this improve, and by how much?"
- "What's the first workflow you'll automate, and what's the ROI?"
- "What happens if the AI makes a mistake?"
- "Can you show me a before/after from a similar project?"
What a good answer sounds like: Specific, conservative estimates. ("This should reduce your invoice processing from 4 hours to 20 minutes. Based on similar builds, expect 15-25 minutes initially, improving as we refine.") Honest discussion of failure modes and human-in-the-loop requirements.
What a bad answer sounds like: "AI will 10x your productivity." / "This will revolutionize how you work." / "Our AI is so good it doesn't need human review." / No specific metrics. / Guarantees that sound too good to be true.
Document checklist: Written success metrics with ranges, not promises. Error handling plan documented. Human review points identified for critical workflows. Pilot or phased approach proposed (not "big bang").
The Complete Vetting Checklist
Before First Call
- Check how long they've been in business (LinkedIn, domain age, incorporation records)
- Review their own website — is it well-built? Fast? Professional?
- Search "[Agency Name] review" and "[Agency Name] complaint"
- Check if they have a real address, not just a virtual office
During Discovery
- They ask more about your business than they talk about themselves
- They identify edge cases and failure modes without prompting
- They give a rough price range before you commit to a scope call
- They explain what they WON'T build (hugely important — boundaries indicate experience)
Proposal Review
- Scope is specific, not vague
- Timeline includes your review/feedback time, not just their build time
- Pricing has line items you can understand
- IP ownership clause is clear and favorable to you
- Exit/termination terms are reasonable
Before Signing
- You've spoken to at least one reference
- You've seen a live example of their work
- You understand exactly what you'll own at the end
- You know what happens when things break
- Payment terms are milestone-based, not 100% upfront
Reference Check Template
Opening: "Hi, I'm considering hiring [Agency Name] for an automation project. They listed you as a reference. Do you have 5 minutes?"
Questions:
- What did they build for you, and is it still running?
- How long did it take, and did they hit deadlines?
- What surprised you — good or bad?
- Did anything break after launch? How did they handle it?
- If you were doing it again, what would you do differently?
- Would you hire them again?
Red flag responses: "It worked for a while but we had to rebuild it." / "They were hard to reach after we paid." / "It cost more than quoted." / "We can't really modify it without them."
Related Resources
- AI Agency vs In-House Developer — The cost breakdown
- Build & Transfer vs Retainer vs SaaS — Pricing model comparison
- AI Automation Cost Calculator — Run the numbers yourself
- 7 AI Automation Case Studies — What good delivery looks like
BluprintCreations — We'll pass every check on this list. Ask us anything.