“Older people can’t use technology”: Refuting assumptions through research 

In the 2000s, as the organization known now as AARP saw the “silver Tsunami” of Baby Boomers entering old age coming, I was fortunate to get to do some research along with Ginny Redish about the experience of older adults as they interact with the world. 

AARP wasn’t the only organization looking at this, though it makes sense that they would. AARP is a nonprofit that offers services to people over age 50 and lobbies Congress on issues related to Social Security, Medicare, and other policies that affect older people and their families. 

Fidelity, a wealth management company with a robust human-centered design practice, realized that many of the daily users of their website were in their 80s. It made sense to understand the experience older people had as they interacted with the information and available transactions online. 

(This was relatively early days for the internet. There were not a lot of transactions you could perform online. While stock trading came online fairly early in the 2000s, exchanging mutual funds – the backbone of most retirement accounts – was not available until 2003 or so.) 

Your perception of what older people are capable is probably wrong

Going into the research, what we heard from mostly people younger than age 50 was that older people couldn’t cope with technology. That they would struggle with websites in ways that younger people would not. 

What we all realized pretty quickly was that there was a huge range of ability, attitudes, and aptitudes in this massive demographic.  But if a website was not usable, it wasn’t because the user was 63 or 89. It was because the website wasn’t usable by lots of people, independent of age. 

We took apart what people meant when they assumed that older adults couldn’t deal with technology and what we found was a medicalized mental model. The assumption was that as the warranty runs out on what were assumed to be perfectly functioning body parts. Then, when you start to experience accelerated aging, you become disabled. You might have poorer eyesight or hearing. Dexterity and mobility degrade. In addition, the mental model of older people held by younger people is that older people also are feeble-minded. Older people are prone to short-term memory loss or just general degradation of cognition. 

As Amy Lee, then the head of customer experience for web at AARP said in her forward to our report: 

“The existing heuristics seemed to me to be focused on people’s disabilities rather than on people’s abilities. Not everyone over 50 has eyesight poor enough to require maximizing the size or contrast of text of a web page. Not every person over 50 has problems with motor control or significant short term memory loss. The diversity of this demographic group is stunning. Not everyone over 50 is new to the Web or afraid of their computer. Why are we trying to lump them all together like that?” 

I was in my early 40s when Ginny and I did this work. I am now in my early 60s. The empirical evidence stands up to what we learned then. I think I’m way smarter now, much quicker on analysis and critical thinking that I was then. This is wisdom (or it could be my own perception through some age-related degradation like early dementia – you tell me).  My eyesight is actually better than it was then. Turns out that the shape of your eyes change over time, and sometimes that is in your favor. Unfortunately, the genes I inherited from my wonderful parents included markers for arthritis. From both of them. I  feel this every day. It does not impede my ability to use the web. 

In 2025, websites, apps, and other technologies are still pretty unusable by lots of people. This is largely because they are made by people who are not their users and because those well-intentioned designers and product people are not learning from people who have lived experiences. And, as technology gains more features and functionality, it comes with more complexity not more simplicity. 

Some takeaways and a model for designing for everyone, including older adults

I’m going to link to all the reports from the work that Ginny and I did, but that’s not the same as directly observing individuals interacting with a thing that someone has designed. 

Our heuristics were informed by also observing older adults interacting with AARP.org and other sites. Among the insights I gained that linger with me today are these: 

  • People perceive “old” as about 20 years older than they are. 
  • Age is a moving target. You don’t turn 50 and fall apart. Different things happen (or not) to people at a range of ages and not all of them are strictly age-related. 
  • If we are lucky, all of us will get to experience aging. 
  • People who are in their 80s and 90s now used computers in their professions during their working years. They may still be happily in their working years. Some of them invented the technology we use today. 

But the big ah-ha that Ginny, Amy, and me had was that there were some simple factors to consider in the usability and accessibility of websites for older adults. At the time, we heuristically placed interactions on sites on scales that we used to try to capture the experience an older person might have. The factors were Age (because AARP), Ability, Aptitude, and Attitude. In our report, we described them this way: 

  • age: including chronological age, but taking into account life experiences 
  • ability: cognitive and physical
  • aptitude: expertise with the technology 
  • attitude: confidence levels and emotional state of mind 

Yes, chronological age is a kind of measure, but one 70-year-old might have amazing skin, excellent eyesight, and be able to run marathons because a combination of genes, privilege, and other factors. Another might have been exposed to environmental, genetic, or other factors that mean their mobility is restricted to being homebound. 

People of all ages struggle with using technology

Later, around 2008,  in some work I did for a company that was a pioneer in online learning, I applied this model to college students who were the audience for the startup’s prototype product. The participants aged in range from 18 to 30 (so-called “adult learners” who maybe were returning to get or complete degrees). What we saw was that age was not a factor at all in usability and accessibility of online tools and websites. 

We met 20-year-olds who were the perfect target audience for Facebook but didn’t know the first thing about how to interact with it, or why you would want to.  When we put them in front of our prototype, we saw no effect for age.  We did see effects for what we (and my friends at Fidelity) called “expertise.” Expertise came from a combination of ability, aptitude, and attitude. 

I have applied the model in formal and informal ways in studies since then and found the same thing: Age is not a factor in how well a design performs for older people. 

In the category of “everything old is new again,” I have had the delightful privilege of sharing my wisdom with lots exceptional designers and product people over the last several years. One thing that is not exceptional about them is that they, too, assume that older adults struggle with technology. So, it’s time to revive this work and get it out in the world again. 

AARP Audience-Centered Heuristics: Older adults (pdf, 106kb)
Twenty heuristics, each with several questions, from the following two studies.

Chisnell, D. and Redish, J. C., 2005, Designing Web Sites for Older Adults: Expert Review of Usability for Older Adults at 50 Web Sites. (pdf 1.8Mb)

Chisnell, D. and Redish, J. C., 2004, Designing Web Sites for Older Adults: A Review of Recent Research. (pdf, 397Kb)

You might also want to read our insights about recruiting and working with older participants in usability studies (pdf, 156Kb)

Human Centered Government Service Delivery

This is a course syllabus for a masters class I taught at the Kennedy School of Government from 2017 to 2020

You’ve actually been making design decisions your entire life. In this course, you’ll gain skills and learn techniques for using design consciously to define problem spaces and to carry out your intent. This highly interactive field course presents processes and practices for applying design to digital government and policy. The activities and assignments in this course will give you tools to understand the lived experience people have with government and how to deliver better services to them and outcomes for them.

This module is a deep dive on understanding user needs through the lens of government policymaking, using design thinking methods and techniques. The methods are best learned through practice. Lecture will be light. Class time will be workshops and activities. Work outside the classroom will be substantial, with at least 2 hours of reading/videos plus 3-5 hours each week of working with your team to complete the course challenge. The goal of the course challenge is for your team to reach and communicate a deep understanding of a social problem to identify opportunities to improve government service.

Course project: Design challenge

All teams will work on the same design challenge. For example:

  • How might we prevent families from dropping out of the public school registration system?
  • How might we increase participation in local elections?
    How might we make public transit universally accessible?
  • What challenges do people with disabilities face in a widespread health crisis?

The challenges emphasize interacting with government and advocates in ways students don’t, ordinarily, and with people they probably have never noticed before. You’ll learn about aspects of the lived experience that you’ve never thought about before.

The goal is to gain a deep understanding of a problem space through design processes and practices. 

Learning objectives

  • Learn the importance of and skills for understanding users’ needs
  • Learn how to define the right problem to design the right thing
  • Learn to tie systems thinking to design of government services
  • Learn to use discovery and research methods to mitigate risk

Before the first class meeting: Submit a 5-minute video 

With this information, I can deliver a better course for you because I’ll have information about your related experience and your goals.  

1. Read/watch these (they are all short):

2. Make a 5-minute video of you responding the statement, “Tell the story of a project you’re proud of as it relates to this course.” Address each of these points:

  • your role in the project and where you fit in the team
  • how you define what a “user” is
  • how you helped ensure that the team you worked with had a shared understanding of the work
  • your process or approach for testing requirements and assumptions (as related to the readings – for example, where did your work fit into the double diamond design process, what steps you and your team did or might have missed in the design thinking approach,)
  • challenges you encountered and how you overcame them
  • what you’re hoping to do differently at the end of this course

3. Submit a link for your video by email to the instructor (dana_chisnell@hks.harvard.edu) by Wednesday before the first class meeting. 

How to get the most out of this course 

Do the readings or watch the videos before each class, please. Doing that will help you understand better what we are doing in class and will help your team move faster. We will not go over the readings or videos in class. You will be expected to have done that work ahead of time. You may be quizzed about the content of the readings and videos.

Come to class ready to learn and practice a technique or method for a step in the process of problem definition. You may be asked in class how to perform the featured method and model it for the class.

You’ll get the best grade if you participate in discussion and activities during class, and make thoughtful and substantial contributions to group projects.

Assignments handed in after the due date will be credited at only a fraction of their total value, unless you ask me for prior approval to be late. I might not grant approval.

I expect students to devote a significant amount of time to their projects (3-5 hours a week). Students are expected to attend every class session, minimize outside class commitments, and make this class a priority.

I do not recommend taking this course while overloading, taking multiple client-based courses, or with an overly demanding schedule. Contact the instructor if you plan to take other high-commitment courses in the same semester such as DPI-663, Tech and Innovation in Government.

Laptops and Cellphones: There will be many class sessions in which you never touch a laptop. However, expect to bring a laptop to class for co-working sessions. Please refrain from using laptops during lectures and workshops, so you can focus and take part in discussion. Cell phones are not permitted in class.

Absences: This is a 6-week course. I expect students to be present for every class. If you must be absent, notify the instructor and your team at least two weeks in advance. Arrange beforehand with your team to make up the work.

Your grade will be lowered by a fraction for every absence. For example, if you would otherwise get an A, one absence will give you an A-. Absences also interact with the quality of your work. See the grading rubrics below.

Food in the classroom: Class meets for 3 hours each Friday for 6 Fridays. Please try to eat before you arrive at class. There will be brief breaks during which you may have a snack, but breaks will not be long enough to get meal-worthy food. Bring snacks with you.

Readings and resources

Course readings include articles, books, book chapters, videos, and blog posts, all of which are posted to the course website.

You must do the readings and use the resources for each week before you arrive at class for that week. Doing so will help you in the class activities and the week’s assignments.

Required texts: 

There are these recommended texts: 

You can order the required books here.

Most of the videos are in the All You Can Learn from UIE. Everyone will use the same log-in information. (See below.) The service is free to students of this course. 

Activities and assignments

In the Course Overview below, each week lays out key topics, at least one activity, and related assignments. We’ll cover key topics and activities in class meetings. You’ll do assignments outside of class time.

Assignments and grading procedures

There are assignments every week.

Final grades will be based on the following:

% of final gradeDeliverable
25%Written reflections every week, submitted in the class discussion area in Canvas, due by noon on Saturday following each class meeting.
25%Written assignments that contribute to your final project. These are due by noon Eastern Time on the Wednesday after they are assigned. Each week, you will work with your team to develop and iterate on plans and artifacts (such as research plans and scripts, stakeholder maps, user journey maps, service blueprints, and other visualizations) that show your growing understanding of the problem space of the course challenge. You will submit polished, professional work worthy of distribution in a large organization to help everyone understand the project.
25%Team work and individual contributions, as well as participating in class. Your peers will also rate your contributions to the project.
25%Final team presentation, which includes a written final report and slide deck.

Assignment and reflection grading

Reflections are graded as submitted / incomplete. You will write a short reflection of what you learned each week. 

Assignments are graded as group or team submissions each week. Each assignment is worth 15 points. 

Completeness  Including items listed in the assignment5.0 
Evidence of work and progress  Showing steps and learning through research, analysis, and reflection5.0
Organization  Showing thoughtfulness in pulling together the pieces to make a coherent whole5.0

The final presentations, report, and slides are evaluated together for Week 6, for a total of 100 points. 

Evaluation criteria for final team presentation

See the description for the last week of class.

Course overview and class schedule

Note: You will write reflections after every class. Each reflection is due to the instructor by noon on the following Saturday.

Week 1, Monday-Thursday (March 23-26)

Yes, this is before the first class meeting. 

Read/watch these (this is the recommended order):

Videos

You must log into the All You Can Learn library to watch many of the videos. This page shows all of the videos in the AYCL library for this course: https://aycl.uie.com/offer/hks20 (Links to an external site.) 

Use this userid and password: 

userid: ocm@hks.harvard.edu
password: HKSDPI-676

Readings

Collective Story Harvest

User Research 

Policy and government services 

Week 1

Understanding users’ needs

The key to delivering government services that are useful, usable, and pleasant is understanding users’ needs. How do you do that? And how do you document and represent those needs in a way that you can track over time?

Class topics will include:

  • Importance of design and design thinking in government
  • Process of design: diverge, converge, diverge, converge
  • What a project looks like: our design challenge
  • Forming research questions and creating a research plan

In-class activity: Collective story harvest related to the challenge question.

Slides: HKS-DPI-676-Chisnell-week-1-2020

Download

Assignments

With your team, self-organize to get this work done:

  • Desk research – learn about accessibility in digital (hint: WCAG 2.0)
  • Draft a rough research plan (1-10 pages of bullet points) that includes 1-3 focus/research questions. Include a draft of your interview/observation guide.
  • Interview 5-10 people with accessibility challenges or disabilities about their experiences related to Covid-19. Remember to collect basic behavioral and demographic data, such as how and if they’re working, how they manage daily activities, what their age and disabilities are, etc.
  • Create one document that includes insights from the desk research, your research plan, interview guide, and insights from your first interviews. Extra points for a well formed, well organized document with clear, descriptive headings.
  • For Friday’s class, write out what you, as a team, think the project research will find the problem space to be. Print it and put it into a sealed envelope. Bring it to class.

Readings (for the next class)

Week 2

Sense making: systems thinking and service design

Government is a series of small systems that work within larger systems. Feeding into the overall government system are dozens of services. Some services talk to each other; some stand alone — or pretend to, or are forced to through poor governance and lack of political will.

This week we will understand how to see a government service in its truer context, how to push beyond pseudo silos, and how to begin widening our points of view to take a more meaningful approach to research.

Class topics will include:

  • Service design blueprint
  • Stakeholder mapping
  • Mapping the user/customer journey
  • Touchpoint canvas

DPI-676_Week2_class_meeting_2020

Download

HKS-DPI-676-Chisnell-week-2-2020

Download

Assignments

Team: Conduct field research. In addition:

  • Revise your research plan, interview guides, etc. to reflect the direction of the research and the research questions you have narrowed down to
  • Conduct 5-10 interviews or observations with MBTA riders
  • Sketch out your service design blueprint (version 1, in class)
  • Create a stakeholder map and interview 3-10 stakeholders (who are not riders)
  • Pull all of the artifacts together into one document and write a narrative (5 to 25 pages) that describes each, what it represents, where your gaps in knowledge/data/research are, assumptions you’re making, questions you still have about the problem space.

Readings (for next week)

Week 3

Problem definition and convergence — how does it feel?

Now you have some of the story of the lived experience. And a lot of qualitative data. How does it feel to be in the problem space?

It’s time to revise the blueprints and maps based on your recent research: refine observations and fill gaps (or clarify what the gaps are in your data, why they are there, and whether you’re going to do anything about them). But before you do that, what could you learn by doing versus listening?

Physical, experiential prototypes of the problem space help us understand nuances of the user experience in ways you can’t just by observing, and validate (or dispel) assumptions that have built up through research.

Class topics include:

  • Role playing and writing the future story
  • Body storming
  • Physical prototyping of touchpoints, especially ones you don’t know well, risky transactions, and core interactions between service and user

Assignments

  • As a team, create a physical prototype that you can literally walk through.
  • Document walkthroughs and discoveries with photos and videos (if appropriate) — show before-and-afters.
  • Update journey maps, service blueprints, and stakeholder maps based on insights from walking through physical prototypes.
  • Add these artifacts, with explanatory narrative, to the  document you developed in week 1 that includes insights from the desk research, your research plan, interview guide, and insights from your first interviews.

Readings and resources (for next week)

Videos 

Week 4

Analyze and synthesize collaboratively

Teams that deliver the best products, services, and experiences are cross-functional and highly collaborative. This week, we’ll practice some simple techniques for quickly documenting feedback from users, coming to consensus the priority of problems found, and for getting to better design direction.

Topics and techniques will include:

  • Design principles
  • Rolling issues
  • KJ
  • Obindi (observation to inference to design direction)

Bonus: Fishbone analysis — https://www.cms.gov/medicare/provider-enrollment-and-certification/qapi/downloads/fishbonerevised.pdf  

Assignments

  • Conduct 5-10 more interviews or observations with people staying home through the Covid-19 pandemic who may or may not have disabilities. Document the characteristics and habits of these participants in your report. 
  • Reach your own priorities using the methods. Show evidence on your journey map, stakeholder map, touchpoint canvas, and service blueprint of issues discovered / resolved, compare what you guessed would happen to what did happen.
  • Document points in the service / product where design has happened, show what seems intentionally designed and why you think that.
  • Review the future stories from Week 3 and compare to what you know now. Does the future story of outcomes change? If so how and why?
  • Create a document, or add to your ongoing report, that includes narrative reporting of the above items. Include a section or table that lists each participant, their characteristics, and their work and life routines and habits. 

Readings and resources (for next week, due April 24)

Week 5

Mitigate risk and measure outcomes

A classic problem in government is risk aversion. This is why most important IT projects have been massive and long: the idea that you could document all of the requirements ahead of time and then build to those requirements has never worked.

Government projects typically are measured based on schedules and budgets. But what if policies were concerned with measuring outcomes for individuals? This week, we’ll look at where design has happened intentionally and unintentionally, and how the difference affects people’s futures. We’ll generate ideas for addressing some the design problems that drive outcomes for individuals.

Class topics include:

  • Envisioning outcomes
  • Success criteria
  • Experience metrics

Assignment

Finish up projects:

  • Update all of your materials, documenting where you started, what you learned, and where you are now
  • Discuss methods and lessons learned
  • Describe what you know about the challenge: Define the problem space and success criteria, as well as how the success criteria would be measured
  • Document design principles for possible solutions
  • Compare what you thought you would find out with what you did find out

Week 6

In-class retrospective, AMA, working session, and walk-throughs 

Come to class and use the time to clarify, catch up, refine, and ask questions. You could even rehearse your final presentations. 

Week 7

Final presentations (25% of your grade)

This week, you’ll present your insights to the instructor and various other audience members. These major deliverables that make up 25% of your grade:

  • A written, narrative report
  • A slide deck
  • Your team’s presentation

Report

Please collect these items together in ONE document and develop a coherent, meaningful narrative with them:

__ The challenge question and a description of how your team approached it

__ Research report (based on the research plan outline)

__ Stakeholder maps (with written explanations of each and the progression)

__ Journey maps (with written explanation of each and the progression)

__ Service maps (with written explanation of each and the progression)

__ Description of the problem space, and how your research and the insights you gained helped you form that definition of the problem

__ Success criteria and measures: What would success look like and how would you measure it

__ Theories that you would propose working on in a new phase to approach closing the problem space

__ Send it to dana_chisnell@hks.harvard.edu by 5pm Eastern Time, <date>

Slides

In addition, as a summary that can stand by itself (without the report and without the presentation), deliver:

__ A slide deck that provides visuals and a backbone for the story of your work


__ Send it to dana_chisnell@hks.harvard.edu by 5pm Eastern Time, <date>

Presentation

__ An excellent, 15- to 20-minute presentation where each person on the team presents an equal part of the story, equally interestingly. Demonstrate that you all have fallen in love with the problem and that you are deeply interested in it.

Evaluation criteria

Your presentation will be judged on these criteria: 

The quality of the story you tell about the problem space. 
Quality of the story you tell, as a team, about the problem space and your understanding of it. Evidence of thoroughness and professionalism. Demonstrate an understanding of your core user’s needs. Apply systems thinking to answering the challenge question
 30 points
Individual knowledge. Evidence that each person on the team has an understanding of all of the research, your data, the models you have created, and the overall problem space. Each person on the team should be able to describe the work, the insights, and the problem space equally and similarly 30 points
Story of progress. Demonstrate that the team took a starting position, explored the space expansively, looked at adjacent possible explanations or ideas, and evolved your understanding of the question, the domain, and the problem space. Define the problem space. Apply discovery and research methods to demonstrate how you tied research questions to methods and insights to describe possible (positive and negative) outcomes.25 points 
Professionalism.
Show a clean and clear organization of the materials, sections, maps, etc. in your written presentation and report, and practiced preparation in your oral presentation
 15 points
Total possible points for final presentation   100 

Peer evaluations

During this last meeting, your teammates will rate your performance over the course. Participation in class, along with individual contributions to team project work make up 25% of your grade.

Revisions

You may get substantial feedback during your presentation. If you would like to make revisions to your report and slides to incorporate this feedback, you have until noon on May 1 to make the revisions and request a reconsideration of your grade for the project.

AIOSEO Settings

Move upMove downToggle panel: AIOSEO Settings

General

Social

Schema

Link Assistant

Redirects

Advanced

Snippet Preview

Human Centered Government Service Delivery | Dana Chisnell

You’ve actually been making design decisions your entire life. In this course, you’ll gain skills and learn techniques for using design consciously to define problem spaces and to carry out your inte…

Page Title

Click on the tags below to insert variables into your title.

 Page Title

 Separator

 Site Title

View all tags →

Page Title | Site Title 

58 out of 60 max recommended characters.

Meta Description

Click on the tags below to insert variables into your meta description.

 Page Content

 Separator

View all tags →

Page Content 

365 out of 160 max recommended characters.

Focus KeyphraseAdd Focus Keyphrase

Get Additional Keyphrases

Additional Keyphrases

Improve your SEO rankings with additional keyphrases.

Multiple Keyphrases is a Pro feature. Click here to get AIOSEO Pro →

Page Analysis

Basic SEO1 Error

TitleAll Good!

Readability4 Errors

  • Meta description length The meta description is over 160 characters.
  • Content Length The content length is ok. Good job!
  • Internal links You are linking to other resources on your website which is great.
  • External links Great! You are linking to external resources.

PageBlock

Human Centered Government Service Delivery

Set featured image

3,522 words, 19 minutes read time.

Last edited 3 years ago.

StatusPublished

PublishAugust 7, 2020 7:36 am

Link/human-centered-government-service-delivery

AuthorDana Chisnell

DiscussionClosed

Revisions

5

ParentNoneDon’t update the modified dateMove to trash

AI Assistant

AIOSEO

Open save panel

  • Page

Please connect your user account to WordPress.com

Notifications

Please connect your user account to WordPress.com

Story-driven experience research on pandemic unemployment

The COVID-19 pandemic shut downs and stay-at-home orders starting in March 2020 put millions of Americans out of work. On March 28, 2020, Congress passed the CARES Act, which, among other things, made billions of dollars available in new pandemic unemployment programs, but states struggled to implement and deliver these benefits. As of the end of July, 30 million people had applied for unemployment assistance, many of them for the first time, ever.

In my role with Project Redesign at NCoC.org, and in partnership with New America’s New Practice lab, I led a team of researchers to interview people from across the U.S. in May and June to learn what it has been like to apply for unemployment and other benefits during the pandemic.

Telling the story of living experts in near-real time

When Tara McGuinness contacted me about doing this project, she was ready to experiment with methods and techniques. She and her colleagues in the New Practice Lab at New America wanted to bring the experience people were having to life for law makers and advocates who usually make their decisions recommendations based on what they see in spreadsheets. We wanted to do a few big things with our approach.

Amplify voices and elevate stories of real people. Those real people are the experts on the experience they’re having — living experts. To convey the experience people had applying for unemployment and other benefits during the pandemic, we documented “thick data” in 2- to 4-page stories. The purpose of those stories was to make the people real for readers, who we hoped would be law makers and their staff members.

Urgently learn about the lived experience. The urgency came from 2 factors. One was that the depth of the needs changed over just a few weeks. As time passed, the experience claimants had shifted from applying to waiting for funds to show up. In the meantime, bills needed to be paid, savings were depleted, jobs disappeared permanently. The other consideration was that Congress would be drafting the next stimulus and recovery bills in May and June. We had a chance to get the stories to people who influenced the design of those bills.

Conduct the research in the open. Show the work. To invite everyone in. Those stories, along with a few slides that highlighted a story or two and key takeaways, went to partners on the project, community-based organizations, media, and to legislative staff on Capitol Hill. We had to give up being precious about how polished the deliverables were. They needed to be just good enough.

We had to give up being precious about how polished the deliverables were. They needed to be just good enough.

It was our intention to invite partners and community-based organizations and legislative staff to observe the interviews. Just scheduling the interviews and conducting them within the time we had proved to be pretty challenging, so we didn’t get to do that. I feel like we know how to do it now, and probably could pull it off for future research projects.

We did develop a research kit and a workbook for anyone who wants to do research like this. You can download the research plan, interview guide, and story template, along with a few other tools for free from the project website.

Share what we learned publicly, as we learned it. Typically, a research team would do the interviews, transcribe them, code the transcriptions, and analyze the coding to identify findings. As you can imagine, this would take a while — for 33 interviews, it could take months. We didn’t have months. We didn’t even have weeks.

As we captured what was working for claimants, what wasn’t, who helped them, who needed help, what time passing felt like, what else people had going on, we wrote up the interviews into stories within a day of doing the interview. We collected interviews together each week, and distributed those collections of 5 to 15 interviews each week.

We were not focused on making policy recommendations. Instead, we documented the stories and drew observations and insights from what we heard.

Doing experience research during a pandemic

There were more than the usual constraints on “field” research in May and June 2020. For example, we couldn’t actually go into the field. It wasn’t safe for the research team or the participants because anyone could be carrying COVID-19.

We also needed to hurry up and do this work because the benefits that were designed to help people were also scheduled to run out. Arranging home visits and traveling takes a lot of time. Congress had set time limits on the first stimulus and recovery bills. We wanted to use what we learned to help the people designing the next set of stimulus and recovery bills make good decisions.

So, it’s not like we could do ethnographic research, really. No access to living spaces. Not enough time. We went into the study thinking like ethnographers, but we needed a few shortcuts. So we borrowed an idea I learned from Kate Gomoll for the interview stories, and some techniques from collective story harvesting. So, we’re calling it experience research, and we relied on the living experts to help us begin to understand what they were going through. We used approaches lovingly borrowed from equity-centered community design, developed by Creative Reaction Lab.

Delivering stories, briefs, and a workbook

The 2- to 4-page stories we wrote after each interview, centering on a few focus questions, were our first deliverables, and we published them each week of the project as we completed interviews. Over 3 weeks, we conducted 33 interviews.

Next, using theme or lenses that we identified ahead of the interviews, we pulled observations and quotes from each interview and conducted some light synthesis. We used that synthesis to write briefs for each of the themes.

We wanted not only to convey insights from these interviews, but also to make the methods and techniques available to others. We hope that researchers in the public and private sectors will pick up inspiration, at least, from how we did the study. We also want to make it easy for organizations that don’t have professional researchers to learn what is happening with their constituents. Our research workbook and kit is available for free to anyone who wants to use them.

The links below open up PDFs. (Sorry about that. Soon we’ll have a beautiful website with fully accessible content.)

Overview

Themed briefs

Doing research like this yourself

About the project 

Full report — Stories, briefs, participants, methods, mechanics, team

Would I work this way again? Yes.

Leading a diverse team of researchers, most of them part-time contributors, to collect a clear and specific snapshot of a civic experience through stories was intense, gratifying, and humbling (in the best possible ways). I learned from everyone who contributed — both researchers and participants. Years later, I carry that experience with me along with the voices of many of the people we interviewed. This project both tested some ways of conducting research and capturing what we learned, and formed a kind of playbook for how I’ve since shaped the work and training of practitioners of human-centered research and design. Key takeaways on approach and practice:

  • Stories and names are powerful. Not everyone in the study allowed us to use their real names, but being able to tell the story of a person’s experience carries a grounded truth that is meaningful and compelling for readers. Attaching a name makes the story a human story.
  • Veracity in insights and findings is shaped by both the people who conduct the research and the participants. The lived experiences of both intersect in the research design, data gathering, discussion, analysis, and conclusions.
  • Breaking the reporting into briefs that were thematic worked well for reaching different audiences. The format of asserting the insight in the headings within the briefs was effective, especially for busy readers.
  • Opening up preliminary findings to subject matter experts and interested stakeholders each week was extremely challenging but was helpful in testing out what we thought we were learning from what we heard in the interviews.
  • I’m proud of building the research kit and methods workbook, and that teams working in state governments picked these artifacts up and ran their own studies. They reported that the methods were effective for not only gaining understanding of the qualitative experience their public was having, but also bringing along partners and stakeholders so they could make better-informed policy implementation decisions.

What can go wrong when we get things right?

I think a lot about how design affects the world. And for a long time, I’ve been thinking about and studying how focusing on one user doesn’t always produce the outcomes we want. Humans are social creatures. We have relationships. Those relationships don’t fade or disappear when we’re interacting with a designed thing. They’re still present as influences.

This goes for users, but it also goes for designers. There’s no lone designer making all the decisions. Even if you’re a team of one, you’re interacting with any number of other people who at least influence the decisions you’re making. Those folks are often making design decisions, themselves.

All of those decisions interact. They originate in personal experiences, training, data, stories.

So, yes, understand your user’s needs. Deeply. Design for them. But look beyond that single user, and the best possible outcome. What happens for other people or organizations? Is it all good? Probably not. What are the ripple effects of getting things right for our core user?

Let’s look at some cases that I put together in a talk for the IA Conference in spring of 2020. Let me know what you think. (I’ll add captions and a transcript soon.)

For more on how data can be weaponized for domestic violence, read this post from Eva PenzyMoog, a UXer who studies this problem extensively.

The epic journey of American voters

Every 4 years, I get a lot of requests to talk about design in elections from the UX and civic tech communities. Watch the talk at Midwest UX in 2018.

Dana on stage in Chicago, showing the path of the burdened U.S. voter

I’ve also written quite a bit about the hurdles that U.S. voters encounter, based on research I did before and while I was at the Center for Civic Design. There’s also a poster that you can download.

Delight resources

A key element of designing for delight is understanding where your product is in its maturity. One way to look at that is through the lens of the Kano Model. You can learn about the Kano Model and our addition of pleasure, flow, and meaning through a couple of sources:

Read Jared’s article on understanding the Kano Model  (8-minute read on uie.com)

Watch Jared talk about the Kano Model (45-minute video)

 

Dana and Jared have both written about different aspects of delight. It’s not just about dancing hamsters. Delight is much more nuanced than that. The three key elements are pleasure, flow, and meaning.

Read Jared’s overview of pleasure, flow, and meaning. (10-minute read on uie.com)

Read Dana’s series  at UX Magazine

 

Design can be used for good, or evil. Jared wrote about a technique that we use in our workshop that he calls “despicable design.” Going to the dark side can reveal a lot about how your team approaches designing its users’ experiences.

Read Jared’s article, “Despicable Design — When “going evil” is the perfect technique” (12-minutes at uie.com)

 

In our workshop, we also use sentiment words to help teams narrow down how they want people to feel or perceive a service. Here are the basics about sentiment analysis. And a piece from NNG about using the Microsoft Desirability Toolkit from which our use of sentiment words comes.

 

Framework for research planning

One of the tricks to making sure that I’ve designed the right study to learn what I need to learn is to tie everything together so I can be clear from the planning all the way through to the results report why I’m doing the study and what it is actually about. User research needs to be intentionally designed in exactly the same way that products and services must be intentionally designed.

 

What’s the customer problem?

It starts with identifying a problem that needs to be solved, and the contexts in which the problem is happening. This is a kind of meta research, I guess. From there, I can work with my team to understand deeply why we are doing the research at all, what the objective of the particular study is, and what we want to be different because we have done the research.

 

Why are you doing the study?

When the team shares understanding about why you’re doing the study and what you want to get out of it — along with envisioning what will be different because you will have done the study — forming solid research questions is a snap. You need research questions to set the boundaries of the study, determine what behaviors you want to learn about from participants, and what data you can reasonably collect in the constraints you have to answer your research questions.

Continue reading “Framework for research planning”

What to do with the data: Moving from observations to design direction

 

This article was originally published on December 7, 2009. 

 

What is data but observation? Observations are what was seen and what was heard. As teams work on early designs, the data is often about obvious design flaws and higher order behaviors, and not necessarily tallying details. In this article, let’s talk about tools for working with observations made in exploratory or formative user research.

Many teams have a sort of intuitive approach to analyzing observations that relies on anecdote and aggression. Whoever is the loudest gets their version accepted by the group. Over the years, I’ve learned a few techniques for getting past that dynamic and on to informed inferences that lead to smart design direction and creating solution theories that can then be tested.

 

Collaborative techniques give better designs

The idea is to collaborate. Let’s start with the assumption that the whole design team is involved in the planning and doing of whatever the user research project is.

Now, let’s talk about some ways to expedite analysis and consensus. Doing this has the side benefit of minimizing reporting – if everyone involved in the design direction decisions has been involved all along, what do you need reporting for? (See more about this in the last section of this article.)

Continue reading “What to do with the data: Moving from observations to design direction”