How to Compare UX Agencies Without Chasing a Universal Winner

Start with the decision your team needs to make

Before comparing agencies, write down the user or product problem that requires outside help. “Redesign our app” describes a deliverable. “Help first-time customers complete onboarding without staff support” points toward a user task and a result your team can investigate.

Describe the decision behind the project: do you need to learn why people leave a workflow, test a new concept, improve a live product, or add design capacity to an existing team? Different goals call for different agency skills and engagement shapes. A studio that is strong at visual product design may not be the right choice for a research-heavy discovery phase; a specialist research partner may not own implementation or long-term interface delivery.

This first step keeps the comparison grounded in your problem instead of agency branding. It also gives each candidate the same context, so differences in their questions and proposals become useful evidence.

• What decision must the team make, and by when?
• Who are the users, and what are they trying to accomplish?
• What has already been researched, designed, or built?
• Which constraints matter: technology, access to users, compliance, accessibility, budget, or internal capacity?
• What would a useful outcome look like, and how could the team observe it?

Give every agency the same short brief

Prepare a concise brief with the product context, user groups, current problem, known evidence, constraints, decision timeline, and expected collaboration. Separate what you know from what you assume. If analytics, prior research, support data, or a prototype already exist, say what is available and what the agency can access.

Avoid prescribing the solution before agencies have understood the problem. You can request a proposal for a defined project while leaving room for teams to question the brief, identify unknowns, and suggest a small discovery step. Compare how each firm handles uncertainty: does it clarify assumptions, propose a way to test them, or move directly to screens and deliverables?

A shared brief is useful even when the project is not fully scoped. It makes proposals easier to compare because each agency responds to the same starting facts. Record any clarification you send later and share it with every candidate.

• Project goal and decision owner.
• Users, tasks, product stage, and relevant context.
• Existing research, metrics, technical information, and access available.
• Constraints, dependencies, timeline, and budget range if one is approved to share.
• What the agency should include in its response: approach, people, scope, assumptions, fees, and open questions.

Set your comparison criteria before proposals arrive

Agree on the criteria before the team sees polished presentations. Start with the decision you need to make, then identify the few conditions that would make an agency a strong fit or a poor fit for this specific engagement. A clear sequence helps: establish required capabilities first, then compare the quality of the approach, team, scope, and commercial terms.

Separate requirements from preferences. Access to people with a particular research skill, the ability to work within a security review, or support through implementation may be essential. A particular workshop format or a familiar software tool may be useful but negotiable. If every preference is treated as mandatory, the shortlist can become unnecessarily narrow; if a real constraint is treated as optional, a proposal can look attractive while being impractical.

Decide how the team will record evidence. A short written explanation beside each assessment is more useful than a rating on its own: what in the proposal or conversation supports the view, and what is still unknown? If formal procurement requires an evaluation scale, define the scale and its meaning before scoring. For other projects, a simple distinction between confirmed, partly supported, and open can keep the record honest.

For a public-sector purchase, check whether a procurement route or framework sets additional requirements. The UK Digital Outcomes and Specialists buyer guide, for example, sets out a framework-specific sequence for preparing requirements, securing approval, shortlisting, and evaluating proposals. Treat that as guidance for that framework, not as a rule for private-sector selection or every public purchase.^[4]

Choose candidates for capability fit

Build a shortlist from the capabilities your brief requires. A project may call for generative research, usability testing, information architecture, complex workflow design, visual design, accessibility expertise, prototyping, or collaboration with engineering. Avoid treating a long services page as proof that a team has relevant experience in every area.

Look for projects with a comparable user problem, product type, operating environment, or delivery constraint. Exact industry matches can help, especially in regulated or specialized settings, but the agency should also explain which parts of its experience transfer to your situation. A fintech case does not automatically establish healthcare experience; a data-heavy dashboard does not by itself prove skill in multi-role enterprise workflows.

The U.S. Digital Services Playbook advises teams to understand what people need, address the end-to-end experience, and test work with real users. Treat it as a useful public-sector reference for the questions behind a digital project, rather than a universal procurement standard.^[1]

Check that the research can answer the decision

Research is useful when it reduces uncertainty that affects a product decision. Ask each agency to name the question its proposed research will answer and what the team might do differently depending on the finding. “Interview users” is an activity. Learning which parts of a complex workflow prevent a particular user group from completing a task is a question that can guide recruiting, discussion, and synthesis.

Look for a reasoned participant plan. The proposal should explain which people need to be included, how the team will find them, and what access it needs from your organization. A convenient group of current customers may not represent new users, people who left the process, or staff with different responsibilities. If access is difficult, ask how that limitation changes the strength and reach of the conclusions.

Ask how the agency will make the work usable to the team. That can include a research plan, session guide, observation opportunities, a summary of findings, and a discussion that connects findings to product choices. You do not necessarily need every artifact on every engagement; you do need a clear account of what the client will see and how decisions will follow.

Match the method to the uncertainty. Interviews can help explain context and motivation, while usability sessions can show where people struggle with a particular interface or prototype. Neither method automatically answers every question about market demand, long-term behavior, or business impact. A careful proposal states what a method can reveal, what it cannot establish, and what further evidence may be needed.

Compare evidence on the same questions

Use the same evidence questions for each agency. Which project is most relevant to your brief? What did the agency do? What source supports that description? What reached users? Which results were reported, by whom, and with what context?

An agency portfolio is a useful first-party account of the agency’s work. A client publication may independently confirm a project, role, launch, or result if it states those details. A review can help you understand one client’s reported experience; a directory score or category rank describes the platform’s listing. Keep those evidence types separate. A polished case study, an award, and a high review average answer different questions.

Compare the strength and relevance of the evidence, not the number of portfolio tiles. One detailed project that resembles your work may inform your decision more than a large gallery with little explanation. Record what a source establishes and what you would still need to confirm.

• Project relevance: does the case resemble your users, product, or constraints?
• Agency role: is the firm’s contribution stated clearly?
• Delivery: is the work a concept, prototype, pilot, or launched experience?
• Outcome: who measured or reported the result, and what context is available?
• Open evidence: which important part of the story remains unverified?

Compare the proposed approach, not methodology labels

Ask how the team will move from the brief to a decision and then to delivery. A useful proposal connects activities to questions: what does the agency need to learn, from whom, by which method, and how will the finding change what the team does next?

“We use design thinking” or “we are agile” says little about what will happen in your project. Ask for sequence, checkpoints, decision points, and what the agency expects from your team. If a proposal includes interviews, ask what participant profile it needs and how recruitment will work. If it includes usability testing, ask which tasks and prototype state are in scope. If it proposes a redesign, ask how research and validation will inform the key flows.

The U.S. Digital Services Playbook recommends budgeting for research, discovery, and prototyping, using frequent milestones, and keeping delivery flexible as a team learns. GOV.UK guidance on working with contractors similarly emphasizes choosing needed skills and knowledge transfer. Both are useful prompts for discussing how a supplier will work with your team; their public-sector context does not automatically govern a private commercial engagement.[2][3]^[1][3]

• What assumptions will the team test first?
• What research activities are included, and what is excluded?
• How will findings be shared and translated into product decisions?
• When will your team review work and make decisions?
• What changes if early research challenges the original scope?

Meet the people who would do the work

A proposal is also a staffing plan. Meet the people expected to lead research, product strategy, design, and delivery. Ask what percentage of their time is reserved, who makes day-to-day decisions, and whether the named people are available during the proposed dates.

Clarify which roles are employees, contractors, partner firms, or specialists who join only for a phase. That structure can work well when it is visible and coordinated. Ask who is accountable for quality, how specialists communicate, and what happens if a key person becomes unavailable.

Check continuity after the engagement. Will your team receive editable files, research notes, prototypes, design tokens, and decision records? Can engineers ask questions during implementation? Is there a handoff or transition period? The right level of support depends on your product and internal team, so compare the arrangements against your needs rather than assuming every agency must provide the same service.

• Who will be assigned, and what work has each person done that is relevant?
• How much time will the lead and specialists spend on the project?
• Who is accountable for decisions, coordination, and quality?
• Which partner or subcontractor roles are included?
• What knowledge, files, and support will transfer to your team?

Include your own team’s capacity in the comparison

An agency proposal depends on the client team as well as the supplier. Someone on your side will need to explain the product, arrange access to users or data, answer questions, review work, and make decisions. Ask each agency to estimate the time and roles it needs from you. A proposal that assumes weekly product-owner decisions is a different commitment from one that offers more independent research and synthesis.

Check whether the people with authority can participate at the moments the project needs them. If compliance, operations, customer support, or engineering must approve a direction, include those dependencies in the comparison. A schedule that leaves no time for internal review may be unrealistic even if the agency’s own calendar looks feasible.

The best engagement model can change with internal capacity. If your team has strong researchers but limited interface-design time, you may need a partner who works alongside that group. If nobody can recruit participants or interpret findings, a proposal that assumes the client will do those tasks may carry more risk than its fee suggests. Record the client-side work beside the supplier’s scope.

Compare scope before comparing price

A fee is only comparable when you understand what it buys. Put each proposal beside the brief and check the activities, deliverables, staffing, schedule, review rounds, assumptions, and exclusions. One agency may include recruitment and testing; another may price those separately. One may offer design through implementation support; another may hand over a Figma file.

Look for dependencies that can change cost or timing: participant access, client approvals, engineering availability, content readiness, data access, travel, third-party licenses, or compliance review. Ask how scope changes are handled and who approves additional work. An estimate with a clear assumption set can be more useful than a lower headline price with unresolved exclusions.

The Digital Services Playbook’s contracting guidance calls attention to defined scope, milestones, delivery flexibility, and transition plans. These are useful comparison topics for any buyer, while the playbook’s contract recommendations are written for U.S. government digital services.^[1]

• Activities and deliverables included in the fee.
• Research recruitment, incentives, translation, or travel costs.
• Number and type of design or feedback cycles.
• Implementation collaboration, accessibility evaluation, and post-launch support.
• Assumptions, exclusions, change process, and payment milestones.

Check commercial terms and ownership

Before selecting a partner, understand the terms that affect how work can proceed and what your organization receives. Review intellectual property, source files, research data, confidentiality, data handling, accessibility responsibilities, cancellation, warranties, and transition support with the people responsible for legal and procurement review.

Ask whether the agency can use subcontractors and whether any project materials or personal data will be processed by third parties. Clarify what may be shown in the agency’s portfolio and whether your approval is required. These details can matter more than a minor difference in a proposal’s presentation.

For U.S. federal negotiated acquisitions, FAR Subpart 15.1 describes a best-value continuum: the relative importance of price may vary with the acquisition, and tradeoff evaluations must state the evaluation factors and their relative importance in the solicitation. This rule applies to federal procurement, not every private agency selection. The practical lesson for any buyer is to decide what matters before reviewing proposals, then explain the tradeoffs against those same criteria.^[2]

Use a scorecard as a record, not an oracle

A comparison matrix can help a team remember why it prefers one proposal. Keep the criteria tied to the brief and write down what evidence supports each assessment. Possible dimensions include project fit, research approach, proposed team, delivery model, commercial clarity, and transition risk.

Set priorities before the presentations. If you assign weights after seeing a favorite agency, the score can become a justification rather than a decision aid. A simple status such as “supported,” “partly supported,” or “open question” may be more honest than a precise numeric rating when evidence is uneven.

Do not combine a review-platform score, an award, portfolio depth, and proposal quality into one homemade quality number. If your organization needs a formal scoring system, define the criteria, scale, and decision rules in advance and keep the written rationale beside the result.

• Criterion: what matters for this engagement?
• Evidence: which proposal, case, reference, or answer supports the assessment?
• Risk: what could prevent the proposed work from succeeding?
• Open question: what must be clarified before contracting?
• Decision reason: why does this option fit the brief better than the alternatives?

Run the same conversation with each finalist

After reading the proposals, hold a structured conversation with each finalist. Ask the same core questions, then leave time for questions specific to the agency’s approach. Listen for how the team reasons about your problem, not just how polished its presentation is.

Ask for a relevant client reference when the project is high risk, unusually complex, or difficult to verify publicly. Prepare questions about the agency’s communication, response to changed requirements, staffing continuity, and handoff. A reference is one person’s account and should be read in context, just like a public review.

A paid discovery or working session can help when the project is ambiguous and the finalists propose meaningfully different approaches. Define the output, participants, time, confidentiality, and payment clearly. Do not ask multiple firms to produce substantial unpaid strategy or design work as a proxy for future delivery.

• What would you want to learn before committing to the full project?
• Which part of our brief do you think is least certain?
• Who will make decisions on your team and on ours?
• What could change the estimate or schedule?
• What would a good handoff look like for this product team?

Verify claims with focused references and examples

When a public case study leaves a material question open, ask the agency for a relevant reference or a more detailed walkthrough. Choose the question first. If you need to understand team continuity, ask about the people who stayed involved. If you need to assess collaboration with engineers, ask how design decisions were carried into implementation. A general question such as “Were you happy?” is less likely to reveal the conditions that matter to your project.

References are contextual accounts, not independent audits. Ask who is speaking, what work they saw, which period they can discuss, and whether the named agency team is the team they worked with. If the reference cannot address a point, leave it open instead of treating silence as evidence either way. For a high-stakes or unusual project, use more than one source where practical.

A proposal discussion can also test whether a case study is relevant. Ask the agency to walk through one decision, the alternatives considered, and the information that informed the choice. Keep the distinction between the agency’s own description, a client-confirmed fact, and your interpretation. You are looking for a grounded explanation and a plausible connection to your brief, not a performance of certainty.

A practical example: three proposals, three tradeoffs

Imagine a product team comparing three fictional proposals for a complex onboarding redesign. One includes a compelling case study but leaves the agency’s role and participant research unclear. Another proposes research and prototype testing but has limited implementation support. A third has the strongest engineering collaboration but assumes the team already understands user needs.

A useful comparison does not declare a universal winner. It returns to the brief: if the biggest uncertainty is why users abandon onboarding, the first decision may be to test whether the proposals can establish that evidence. If research is already complete and engineering coordination is the bottleneck, the third offer may deserve a closer look. The second may fit if the client can cover implementation internally.

This example is illustrative. It shows why the same agency can be the right partner for one constraint and the wrong fit for another. Record the tradeoff and the evidence behind it so stakeholders can challenge the reasoning before a contract is signed.

Write down the decision and what remains open

A short decision record keeps the selection explainable. Note the project need, candidates considered, evidence reviewed, proposed scope and team, commercial assumptions, unresolved risks, and why the selected option fits. If the decision changes later, the team can revisit the reasoning rather than reconstruct it from meeting notes.

Keep uncertainty visible. “We have not confirmed the case’s launch status” is a useful open item. It should not become a negative claim about the agency or disappear into a general impression. Resolve the questions that affect scope, risk, or fit before signing; keep lower-impact questions in the engagement plan.

After recording the decision, turn any unresolved issues that affect scope, fit, or risk into explicit questions for contracting or the engagement kickoff. A useful rationale should also be understandable to stakeholders who were not in every presentation.See guide to choosing a UX agency · case study evaluation guide · source standards.