Building Products That Users Trust
How apprehensive are you when using generative AI for your work? This capstone project for Stanford Capstone proposes to answer that question.
Implementing a generative AI-powered Broker Assistant that addresses user apprehensions associated with AI adoption, automates complex insurance brokerage tasks, and improves the broker’s decision-making process.
Wireframe for proposed AI-enabled ‘broker assistant’ displaying a comparison for five insurance quotes.
Situation
Professionals in regulated industries, such as insurance, often conduct comprehensive marketplace audits to advise clients on procuring products and services. These audits involve analyzing non-standard attributes and continuously evolving data, making informed decision-making challenging.
Challenge
Despite the potential benefits of AI-enabled tools in assisting with these tasks, there is significant user apprehension. Professionals fear that AI might replace their roles and lack confidence in AI’s ability to handle non-standardized, complex data, leading to resistance in adopting such technologies.
Question
How can a generative AI-powered assistant be designed to automate complex brokerage tasks while addressing user apprehensions and enhancing decision-making confidence?
Complex tasks automated
- The Broker Assistant automates the analysis of non-standard insurance quotes, reducing the manual effort required in marketplace audits.
- By handling complex data comparisons, the AI enables professionals to focus on higher-level decision-making tasks.
Enhanced decision-making
- The AI provides comprehensive analyses and recommendations, augmented with community feedback, aiding brokers in making more informed decisions.
- Real-time data processing ensures that brokers have access to the most current information, improving the accuracy of their advice.
User apprehensions addressed
- The design includes user control features, allowing brokers to adjust AI assistance levels, thereby building trust in the technology.
- Transparent feedback mechanisms are implemented to clarify AI intent and benefits, reducing fears of job displacement.
Outcomes
User Research Findings
Identified that lack of understanding of AI’s intent and unawareness of potential benefits are primary factors contributing to user apprehension.
Design features
- Wireframes illustrate user flows that balance automation with user control, including settings adjustments and proactive assistance features.
- Ethics and societal risk assessments were conducted, resulting in 12 guiding AI design principles, 15 identified risks, and 28 mitigation strategies to ensure responsible implementation.
Conclusion
By thoughtfully integrating generative AI into the brokerage workflow, the Broker Assistant enhances efficiency and decision-making quality while addressing user concerns, ultimately leading to higher adoption rates and improved professional performance.
