A marketing automation platform (MAP), customer engagement platform (CEP), or email service provider (ESP)? Oh my. Those are all the exact same thing, by the way.

After 20+ vendor evaluations, here's what most RFPs miss. So I wrote the guide the vendors never will.

Five things I look for in every customer engagement platform evaluation that never show up in a feature comparison:

1. Don’t accept the RFP as-is

Successful implementations depend on internal alignment as much as the technology itself. RFPs are useful as a bill of materials (pricing tiers, connectors, channels, usage limits, product names), all of which show you what the vendor sells, but not actually why or how it matters for your business.

For ESPs, map pricing to your actual sending program. A contact-based model looks simple until you account for transactional messages, multiple brands, new channels, growing segments, and trigger volume. Then it looks very different.

My favorite move: translate the RFP into plain language based on your business's specific context. Sometimes AI can be helpful here, sometimes not. You need to be able to explain what you're buying to your engineering, your finance team, and your leadership team, and that explanation is your business case. The best technology is unlikely to meet your expectations if you can't build that case internally and tie it to the problems the business is actually trying to solve.

The best vendors will help you do this. That tells you something too.

2. Evaluate for complexity, flexibility, and the long-term roadmap

A feature checklist tells you what a platform does, but it doesn’t tell you whether it will work with the rest of your stack or how much choice you’ll retain after you sign.

I look for interoperability, flexibility, and open APIs. For ESPs, interoperability is mostly a customer-data question. How do identity, consent, events, and channel preferences flow through the platform?

I want to know: 

• Can this platform work with our existing architecture? 

• Can we integrate it with the tools we need?

• If we want to use another provider for SMS or a specific use case, can we separate that out?

Also, ask about migration and implementation scoping before you sign. What does the vendor require from your team to go live? What's a realistic timeline, and what are the most common reasons implementations run over? The demo environment is never the production environment, and the gap between the two is where most surprises live.

One thing most people skip: who will actually run this platform day-to-day? Do you have someone with domain expertise to manage what you're buying? Sometimes the problem has nothing to do with the technology and everything to do with who's operating it. 

Case in point: I've inherited Pardot, Marketo, HubSpot, SFMC, Iterable — you name it — where the previous owner clearly didn't understand its nuances or how to properly implement the platform. 

That kind of mistake costs far more in the long run: a data model, a set of workflows, and assumptions that nobody understands. In an AI world, your data model and context are everything. 

3. Verify their incident response plan

Every platform will experience an incident in which either the entire platform or a specific feature goes down. Even the best, most robust vendors have outages. That's normal. What matters is what happens next, as well as whether the contract includes a strong service level agreement (SLA). 

I don’t care about things going wrong nearly as much as I care about the vendor's response when it happens.

I follow status pages to track real incidents before any evaluation conversation. That way, I can point to specific examples, ask how customers were impacted, and ask how the team managed remediation. It's my favorite tactic for cutting through the sales layer.

I've had vendors put the blame on the customer. I've had vendors explain exactly how they handled the entire process. The difference is very telling.

Here are a few questions I like to ask:

• Who contacts us when something goes wrong?

• How do they contact us?

• How frequently will we receive updates?

• What will you tell us about the incident?

• What does remediation look like?

• What happens after?

ESP incidents affect customers directly. A broken send, a duplicate campaign, a delayed password reset, a consent issue: these create immediate customer and deliverability risks. Risk mitigation and accountability aren't a bonus conversation; they belong in the evaluation before you sign.

4. Go beyond vendor references

Vendor-sourced references are useful. They're also curated. I always find a few of my own outside the vendor's list — usually by searching their customer library, LinkedIn, or Slack communities and asking trusted peers directly.

What I'm looking for is the depth and breadth of how customers actually use the platform, not just what the capabilities promise. And this is where you need to distinguish between a mature lifecycle program and a simpler implementation. A company running newsletters and a few triggers is a different reference point than one running cross-channel orchestration at scale across multiple languages.

Here’s a list of my favorite questions to ask:

• What do you actually use the platform for?

• What do you use other tools for?

• What did implementation require from your team?

• What surprised you after signing?

• When something broke, how did the vendor respond?

I'm not trying to pull a gotcha on the vendor. I'm trying to understand what it's like to live with the platform after the sales process ends.

5. Read the contract yourself

It sounds so basic, but actually read the contract thoroughly before you sign, not after. Renewal clauses, price escalation, seat limits, and overage triggers are the things that bite; they always feel irrelevant until they aren't. Don't rely solely on legal to catch these. You know your sending program better than anyone in that room.

Closing thoughts

An ESP RFP is not just a software purchase. You are choosing the system that determines who receives a message, on which channel, at what time, and with what customer data.

The goal is not the longest feature list, the slickest deck, or the cleanest demo. It's understanding what you're actually buying; how it fits into your architecture, what it will require of your team, how the vendor responds when things go wrong, and whether real customers validate the story you're being sold.

Signing the contract is only the beginning, because you have to live with the decision after you sign.