Introductions

Larson: 

Hello and welcome to the Office of Civic Innovation’s Learning Lab: Digitizing Government Services. My name is Mat Larson and I'm an innovation strategist for the Office of Civic Innovation. I lead the office’s Learning Labs event series. Learning Labs is an initiative which seeks to build the city's problem solving capacity by sharing learnings and methodologies developed from our public-private partnership programs. Today we'll be exploring the learnings and methodologies from our collaborative housing project DAHLIA, which comes from our Civic Bridge Program, a cohort based program that connects city departments with pro bono teams to tackle critical civic challenges DAHLIA transformed the complex paper-based process for applying for affordable housing into a user-friendly streamlined portal for residents today. I'm excited to be joined by members of the fantastic value team: Barry Roeder from the Mayor's Office of Housing and Community Development (MOHCD) and Roshen Sethna and Wendy Fongfrom Exygy. Thank you very much Roshen and Wendy for taking the time to speak with us today. Before we dive into the presentation, I'd like to take a moment for everyone to introduce themselves with their name, title, organization, and share what attracted you to working in the civic space. Roshen, I'm gonna start with you. 

Sethna: 

Great. Thanks for having us, Mat. My name is Roshen Sethna. I'm one of the Partners at Exygy. We're a digital innovation studio that works with the social sector. We worked with Barry and his team at the Mayor's Office of Housing and Community Development to design and build out DAHLIA. We continue to partner with them today as well. I lead all of our work at Exygy in the civic sector, so a lot with local and state government, and I'm really passionate about people getting access to basic social services and safety net work in a previous life. I was also a housing counselor, so this is a really great thread for me throughout my career. I'd love to pass it to my colleague, Wendy. 

Fong: 

Hi, everyone. I’m Wendy, design lead at Exygy. My work as a designer is grounded in the passion for civic tech and how design and technology can raise this all up. I always like to say that we are all government because it affects everyone in my previous life. I was an adjunct professor at City College and also a fellow at Code for America. 

Roeder: 

Thank you so much. My name is Barry Roeder and I manage strategic projects for the Mayor's Department, Mayor's Office of Housing and Community Development. And I just want to say that I am very honored to be part of this conversation today. I'm sort of the tip of the iceberg, if you will. I want to give a shout out to my colleague, Michael Solomon and also Ashley Myers, who were a part of our three Musketeers in the early development of this. And then also the great folks at Digital Services that are just so key to us. And the fact that we, again got this off the ground because of the support of the Office of Civic Innovation. So thanks to all your folks. 

Larson: 

Excellent. Thank you all for the introductions. And joining us again just, can we move to the next slide, just walked through the agenda real quick. Excellent. So we're going to be looking into the DAHLIA case study tackling the approach that was used looking at the user centric approach as well as how that led to the systematic redesign of our housing or the affordable housing portal for San Francisco, we'll have a Q and A portion. We have the chat open throughout the event. So please, as questions come to mind add them to the chat. We'll get to as many as we can any questions that we don't get to in the Q and A portion we'll add in the follow-up email. Additionally, all resources materials shown in this slide will also be included in that follow-up email and this is going to be recorded. So this will, the full video link, will be included in that follow-up email as well. And with that, I'm gonna pass it off to I think Roshen or Barry for a deep dive into the DAHLIA case.

San Francisco’s DAHLIA Affordable Housing Portal: A Case Study

Roeder: 

Sure. Thanks so much. Let me queue up my screen on my end here. So just a quick note that the Mayor's Office of Housing and Community Development as you might imagine has been aware of the challenge for housing applicants for some time. And it's something we tried to solve before but it's a big lift cause there's a lot of moving parts. So we'll talk a little bit about that, but I just wanna again, commend my colleagues for having me to do this before and, and utterly they had to get through. And then so grateful that we had some progress this time around the next slide, please. So there in 2015 then Mayor Ed Lee made a goal for San Francisco and said that by 2020 he wanted 30,000 new housing units and 10,000 of which would be permanently affordable. Just so you know, cause we're there now, we don't have completed numbers for the end of the tail year, but it looks like we made it by the way. But the question is how do you find that? 

The Problem

Roeder: 

All of those wonderful new units fell into the same, an ecosystem that we had problems with forever and see all these fantastic sites on their own on the right. Well maybe with the exception perhaps of the one at the top of the screen, which was our old website. The issue is that finding affordable housing was a matter of luck. My friend’s a housing case manager or a social worker, orI was walking through the day and I walked down and I saw a project that had MOHCD on it. I think it's going to have affordable units. Let me find out about that. So you have to sort of luck out of the draw to find out about them. And it's not clear whether or not you'd apply, excuse me, whether you would be a fit for it. And the information that you get again is just kind of a hit or miss next lot. 

Disparate housing portals created inconsistencies and confusion throughout the search process.‍

So, and then if you decide you, okay, I think I'm going to take a shot at this one. I think I might be a fit. Typically there's long applications that could be up to 12 pages. That's the worst that we've seen plus supplemental information like tax returns pay stubs things like that. And then when it's time to sort of fill them out, as you can see, it could be a long line to drop them off at the developer. And it's all for a very slim chance of placement. And another note too, is if you've gone through this effort to fill all this information out and you happen to have heard about a second or more opportunities, you can't just stop off the information that you've already completed at those other places because they have their own forms and their own process. 

And I just want to underscore that it's, it really is an opaque process, so great. I submitted in my application what happened. We'll let you know if anything happens. So again, kind of about kind of a clunky thing. Quick note in the lower right of this screen is a frightening artifact that was in use until frighteningly recently. And that is our old carnival ticket drum, which by the way, we used to have tape put tape over the door because it would flop open, but basically we would give people a red carnival ticket, which we would drop the other half in this bucket and spin it and pull out of there. Not exactly 21st century processes. We also had concerns about our professional partners and particularly housing counselors you know, and developers and property managers sort of both sides of the aisle. If you are wondering sort of, wait, how am I going to be impacted by this? So it was important for us as we considered the user, which we'll talk more about in a little bit that users were more than just housing applicants. 

A carnival ticket drum was used for each housing lottery.‍

The Solution

Sethna: 

I'm going to talk just a little bit about what we built. So we'll talk about this throughout, but this project started as Mat mentioned as a Civic Bridge project where a pro bono team from Google came in to do a lot of research. And then we as exit G won an RFP after that and continued the work. So a lot of what we built was built on top of the initial work that the Civic Bridge team did. But the solution at the end of the day is called DAHLIA stands for the database of affordable housing listings, information and applications. It's also the San Francisco city flower and the site can be found housing.sfgov.org. I'll just walk through a little bit of the solution. We won't walk through everything, but just to give folks a sense for, especially those who haven't seen it yet. Like Barry mentioned, all of the sites that exist in the city side or the nonprofit ecosystem are really text heavy. 

Check out this DAHLIA demo clip here. 

It's hard to determine if you're eligible for something. So we really simplified everything for our end users who are our housing applicants. Today DAHLIA actually includes both rental and home ownership listings, but I'll just go into the rental to show you how we design these listings. So this is what we call the browse listings page. You can see open listings that are taking applications right now. And you can also go down and see past listings that have closed, but have upcoming lotteries that we'll sort folks into a ranking system. We designed this page really well to surface three pieces of information that people were using to search for housing. So when we talked to a bunch of housing applicants, we saw that mostly they were looking for, what are the unit sizes, what income range do I need to be to qualify? 

And then what's the rent? Those were the initial three pieces of information that people were searching for. And we found that through a research process that Wendy we'll go into later. So we surface that here and if people are actually interested in applying for a listing they can see more information about it. So this is what we call our single listing page or our listing page. And there's a lot more information here about the eligibility criteria, the features of the building itself, the neighborhood it's in and any kind of required documents or rules of the building. So this page is much more storytelling format. It prioritizes information people want first, which is again, eligibility criteria. And in this sidebar on the right side this is what we call a dynamic sidebar, or it just basically changes over time right now, since this listing is open, the main action in the sidebar is to apply online. 

But once the listing was closed, the lottery results would appear here and things like that. And so then if you do want to apply and you click on apply online, we've built the step-by-step application process for folks to apply online. This part of the site is actually human translated into the city's languages. The rest of the site is using Google translate. But you can click into a step-by-step application process that has these five steps. We give you a lot of contexts, we carry you through the process. So we give you documents. You might need to help fill out this application. And then it's just a step-by-step process. So you know, people go through and the great thing about online is, you know, you can build in eligibility rules into the system, right? So one of the rules is that the primary applicant has to be over 18. 

If I put in a birth year, that does not make me 18 it'll flag that for me. So throughout this whole step-by-step application, we have different flags to help make sure people who are applying are actually eligible and that way City and County staff get eligible applications coming in to them. Two other things I'll mention is that DAHLIA is accessible. So these contrast ratios that you're seeing make it very readable for people who have reading issues. We also DAHLIA works really well on mobile. So if you know, shrink the screen, everything will shrink. So people, a lot of people can apply on their phones. And this application process now takes 15 minutes online versus the hours that it takes to fill out what barrier mentioned and standing in line, all that kind of stuff. 

And the last and final thing is that Barry did a really great small political effort to get all of the housing developers to agree on one application process. So now every building in San Francisco uses the same application. That's sort of like a common app, like a college common app. And we really worked closely with that group of stakeholders, all the housing developers to design an application that would work for all of them. So we brought in a lot of those stakeholders throughout the process. So let me go back to our full screen and just show you at the end of the day, when you submit an application, since we didn't go through all of it, just what it looks like at the end you get your housing lottery number. You're able to save that and you can look up your housing lottery rank on DAHLIA after that's posted. You can also save your lottery number on Dolly itself. DAHLIA has an ability to create an account. It's optional because we wanted to make sure that people without email addresses and about, without the ability to create an account, still we're able to submit an application. But if you do want to save your lottery ticket number or save a draft application and things like that, some of those additional features are available if you create an account on volume. So it's a really quick overview of the actual product.

The Impact

Sethna:

We're really proud of the results. Over 650,000 folks have applied on DAHLIA since online applications were available, that was back in 2016. Like I mentioned, the application process. Now it takes 15 minutes online. This is the number I think I'm most proud of is this 97% adoption rate. So whenever you're moving things digitally, you're, you know, you're worried that people won't adopt the new product. People still want to do it on paper. We have a really high adoption rate. So 97% of people who apply for housing apply online through DAHLIA instead of through the paper, which at least pre pandemic paper applications were still available. They married the online application, but we still make them available for people if they need them. And this adoption rate was increased over time. It didn't start out at 97%, but as we learned more and more what people wanted and continued to tailor the system to housing applicants we inched it upwards DAHLIA places, 50 households a month on average in housing. And we've had about 1.6 million site users, so we have really heavy traffic. 

I'm going to hand it off to Barry to talk a little bit just about the history of the approach that MOHCD had been taking and how this new kind of Civic Bridge approach that we were a part of. And mostly that was a part of kind of shifted the approach we were taking to build this product. And then we'll hand it off to Wendy to talk more about that. 

Our Approach

Roeder: 

Yeah. Thank you guys. And thanks for the intro again. So as I mentioned before, most of them have made attempts at this problem again, knowing that, gosh, it's really hard for people to find these things and application is all over the place and it's really opaque. Okay. You know what? We know that we know what the problem is. I'm going to come back to that in a second. So one, we know what the problem is too, you know, let's just put some kind of system, let's get some software folks in here. I don't know. Let's, let's see what they say. Okay. These people, they like Salesforce, which we still use by the way. Okay. And we can make some interfaces and Oh, okay. Well we need to do, to build a database first. All right. Well, let's make a list of all the things that we need to do. 

We need a database, we need an application, we needed this, we needed this. And Oh. And my colleague says, okay, so I, okay. So it turns into a really long list of things we need to do. Let's get going well, we're already in trouble and we're already in trouble for two reasons. One of which is that, as I mentioned before, we know what the problem is when we say that. And, and I think those of us in the public sector are doing this work because we're passionate about creating good in the world which is a really fantastic thing. And I'm delighted just to be sharing this space with my colleagues in that sense all of us, and I include by the way, XOG in this too, which is a B Corp and mission driven, and we just couldn't have done this without had their hearts being in the right place, so to speak. 

So, the first problem though, is that we assume that on behalf of our constituents, we, we, I know what it is, definitely what it's like. Well, wait, wait, wait, wait, stop. Have you applied for affordable housing? Well, no, but I, Oh, okay. Wait a minute. Have you lived in one of our fertile housing units? Well, no, but I, okay. Okay. Well that means already where there's a disconnect. So what we knew, what we learned from our Civic Bridge engagement with some great folks at Google, one of whom by the way, is a current digital services employee. We, we, we took it from the dark side, so to speak and what we learned further with some great teaching from XOG was we'll wait, no, no, no, no. You really see, you really got to put the user and center to your process, even if you think they know what they want. 

And then the other thing too, is that, that big old honking list of stuff that you created to do, that's just not sustainable. I like to think of this as sort of like, my analogy is you know, when all of us were in college or whatever, if you had a semester project, you wouldn't dare not deliver anything that was complete. Like, you know, no, no, it's got all, everything it's perfectly typed or whatever, for those of us that are old enough to type these things everything's all put together. Okay, great. And we were asked to do a different approach and I'll shift it back over to Roshen to talk a little bit about what, what we were taught, which was great. 

The Agile Way

Sethna: 

I'm going to jump in here to talk about the process. So yeah, like Barry mentioned the kind of older school way of building and designing software is that you write out all of the requirements that you need as an organization, as a department or an agency, you think, okay, I need these like 10 things to exist. Usually the list is longer than 10. It's usually not 50 or 80 or a hundred things. And then you spend a few years building all of it and then you release it and you might have built something that people need, but you might've not. And you, haven't kind of checked in with people throughout the process and done kind of smaller, more iterative releases. So what's on the side. Now, the agile way for those that are a little bit newer to agile is about a methodology that helps you release something small early on and test your hypothesis to see, Hey, is this people what people actually need to apply for this service or to get to this goal and then slowly building out the product or website over time. 

So we use the analogy of a skateboard to a car releasing something small that is a skateboard first. And a skateboard is something people can use. They can get from point a to point B. They can't maybe get there as fast as they want. But it's, it's a usable product. And then building out the product over time to layer in different features or different abilities for people to do things. And then at the end of the day, if you do that and you're checking in with people throughout, you're hopefully building the car that people actually want and will actually use. And again, that relates to that 97% adoption rate that we have for the project that we really released, something small. We slowly built it out over time. We were talking to stakeholders and years there throughout the entire process to make sure what we were building was working for them. 

DAHLIA Roadmap

Just to give a sense of the roadmap for DAHLIA. We've been working on DAHLIA since 2015. We handed off this project to Digital Services that is now leading it, and we continue to work with them on it to build it out. A website and a project are usually never done because there's always changes and things you want to respond to over time. These are just some of the features that we've built out over the past five or six years. There's a bunch more in there that I didn't highlight, but just to give you a sense of originally our skateboard that we built and released really in three months. So the Civic Bridge Prop Project happened the summer of 2015 Exygy won the RFP and started the project towards the end of the summer. At the beginning of the fall of 2015, we handed off from the Google Civic Bridge team. 

And we continue to do some design work and build work. And we essentially launched an alpha project or alpha website in three months. And that was our skateboard. And our skateboard was putting up a small subset of the rental housing listings. So what's called the below market rate rental housing listings. That's just one portion of all of the rental housing listings. And we just put up the listings, you weren't able to apply online. You weren't able to keep creating an account. There weren't a lot of tools for city staff to manage the listings as they were coming in. It was literally just a better form of listing so that people could actually browse them and understand what was available to them. And then, you know, at that time, the application process was still on paper. Everything else was still on paper. 

So we just did that piece of the system. Then we said, okay, now people want to be able to apply for these listings online. So in the fall of 2016, after about six to eight months of more development, we released a feature that allowed people to apply online. And that feature took a little bit longer than three months, just because, like I mentioned, we had to make sure that all the housing developers were okay with the one common application. So there's more stakeholder management and things like that involved. And then in 2017 we moved on to more rental listings. So in other types of rental listings or the multifamily, or a hundred percent affordable units, we started to get those online and, and you were able to apply to those. And then in 2018 we built more tools for city and County staff and released agents that were seeing these applications come in and they needed to lease up the building. And as recently as 2019 SF digital services helped release the homeownership listings online. So again, things were iteratively released over time. We tackled different pieces. We didn't try and do everything at once throughout the entire process. We were talking to users and stakeholders to design with them. And then Barry, I don't know if you want to hit on some of these key points as well. 

Agile Approach Key Elements

Roeder: 

There's a couple of the benefits that we got out of this process. That again, you might not be thinking about it. And yeah, so I just wanted to share this with you. So air coverage is one of them. What are you, what are you talking about? So we do live, of course, as public servants, we do live in, I'll borrow from Roshen for a second in a capital P political environment. So, you know, there are certain things, I mean, I, I think probably won't shock you, but I think that mostly staff by large would love to have, you know, seven times the resources to build 20 times the housing. But, that may not be the will of the people as it were, or the will of some of our political stakeholders, et cetera. So there's certain things we can say. And can't say, and I'm sure you as public servants, don't know exactly what I'm talking about. 

One of the great values about having an external partner. So whether it's a Civic Bridge private organization that comes in again for a limited term and gives you some thoughts, or whether it's the folks at Exygy, who've been so fortunate to be engaged with, for the length of time we've been been engaged, you know, they can give you not only a different perspective, but some opinions that maybe we couldn't say otherwise. So we can say, well, see, this is what the findings were. And that's really, really helpful, especially when we're taking a risk to try a new approach, to try new technology. It's really valuable to have the perspective of someone else to validate what you're thinking or again, support procurement is another piece. So imagine the idea of agile is to do work in typically two week sprints. That's what they're called. So in their software that helps you coordinate what are we going to try and do in the next two weeks? 

Imagine us having a conversation with the controller's office and they say, well, how much money do you need or with our Chief Financial Officer at MOHCD, well, how much is this going to cost? We don't really know. We know how much we need for the next two weeks, if that good that it's pretty hard to make that fly, but we did. And we were helped with our partners to do that because as Benjamin McCloskey, our Chief Operating Officer says it's, he knows that he's going to get something out of every single two week sprint, another piece of skateboard and other skateboard and independent definitive point. It stops working. He can reevaluate it versus, okay, I'm going to put tons of money into it or something. And I won't see anything for many months, and then I don't know if it will work. 

Again, this is another sort of benefit of this process. And finally, I just want to underscore this again, it is so important and each step of the way, each one of those sprints to step two, to check in with all your users and in our case, not only housing applicants, but make sure we had a current conversation. I convened what was called the DAHLIA task force that had housing counselors and housing advocates, and applicant proxies on one side of the table, if you will. And housing developers and property managers and leasing agents on the other side. And that was really valuable for us. So I'll turn it back to Roshen and Wendy. 

A Systematic Redesign

Sethna: 

Yeah, I wanted to dig into how we did all of this and we'll try and get through to see you guys have time to ask us questions. But handing it off to Wendy to talk about the exit zone, process and approach that we brought to the table and really MOHCD and Exygy really embedded as a team together to apply this process to the DAHLIA website. 

Fong: 

I want to emphasize that DAHLIA is a fundamentally different system than what existed before it. And it wasn't just a matter of scanning a paper form in order for us to be successful. We had to take a step back and look at the whole ecosystem of affordable housing in San Francisco, starting with MOHCD and three buckets. 

The first one is understanding their services and information, affordable housing opportunities and counseling resources. The second bucket, the people that they're serving that includes external users, people seeking affordable housing, CBOs, housing counselors, and internal users, the MOHCD team and subject matter experts. The last bucket delivery tools and touch points. These are the things that support people's goals and tasks such as the website and the front desk reception newsletters. Exygy’s approach is to connect the dots with these different viewpoints. 

Exygy’s continuous design process.

Now I'd like to dive into the nuts and bolts of our design process at a high level, we're working in a continuous cycle of research and learning, create an MVP or a minimum viable product or prototype that's a skateboard, and then test and iterate. 

Okay. And we're always starting with research and learning. And Barry, would you like to talk about the product requirements document (PRD) from Civic Bridge? 

Research & Learn

Roeder: This was one of the documents that our original Office of Civic Engagement with Google processed for us as a roadmap. Where do we go from here? 

Fong: In this slide, we can see a sample of the amazing work that the Civic Bridge team had already conducted interviews and field research that we were inheriting. And we just kept that we kept that going. We're interviewing folks every few weeks for the next couple of years throughout the project and the digital services team recently did another research round. That's to say research never ends. 

Sample of Civic Bridge’s interviews and field research.

What do we do as part of the research and learning phase? Everything we're doing is research based. So all the data points that we're collecting qualitative and quantitative inform the larger picture with our people driven focus, we're diligent about who to include in our design process. We're not, we're not the experts, right? You're the experts. So we need to build those relationships, understanding that those relationships take time, but also being mindful of most of these team capacity. And everyone's already busy schedule knowing that the payoff is invaluable folding in these rich perspectives throughout the project. So some of the activities that we did during the research and learn phase, we conducted many interviews with internal stakeholders, subject matter experts, support staff, the front desk and external stakeholders, of course like housing applicants, counselors, and developers. And we did a bunch of participatory participatory workshops. These are hands-on sessions with curated exercises, with the MOHCD team and others in the affordable housing ecosystem to truly co-create many early prototypes. And in the photo here, this is one of our first workshops on how you might display the listing fields and coming up with different ideas for the short form application. 

PRD document highlighting problem areas. 

Journey Map

A tool that we used for most of our projects is called a journey map and a customer also known as a customer journey map. This displays the client experience over time, capturing the main touch points for every goal and task. And we're also capturing the emotions and observations involved. So, in the image above, you'll see a sample of an online shopping experience from A to Z. 

The image above is showing you what it more commonly looks like. And you'll see at the top each phase of a customer's journey. So all the steps that they're taking to achieve their goal or task, and you'll see below that the different activities that are performed at each step, also the emotions at each step, their overall experience and any supporting technology or support staff internally to support that experience.

DAHLIA’s early applicant journey.

The above slide shows what we did for MOHCD. This is a very early applicant journey with all the steps. So this is a huge document. And as you can see, we don't need a lot of fancy tools. These are just boxes and arrows to demonstrate the applicant's journey and in the little bit hard to see, but in the yellow you'll see the phase of browsing a property listing to the green section, filling out an application. The blue section is entering the lottery. The purple is reviewing eligibility, and then eventually signing the lease and moving in. Also on this journey map, you'll see the timeline like estimated days for, for all these things to happen. And this was very much a work in progress and a collaborative live document. We shared and embedded this journey map with internal stakeholders and the team to make sure that we were getting it right. 

Create an “MVP”

With everything that we gleaned from our research and learn phase, it informed our next step, which was creating an MVP, the minimum viable product or the skateboard and the prototype.

And we, you know, we're taking all the lessons that we learned from the workshops to create that, that thing that we share with others, real users to get their honest feedback. Every time we did a user test with the prototype, we took that information, shared it with MOHCD teams, the regular meetings, and check-ins pictured. Here is another workshop that we had at city hall to inspire some early prototypes. And in the next slide, you can see the evolution of an early wire frame, some boxes and arrows to give them an idea of how a website layout might look 

Test & Iterate

Okay. And with our prototype, we are continually testing and iterating understanding that we want to be sure that we are sharing it with the community. And this includes non English language speakers older like seniors non non-tech people, if you will, to understand their needs and challenges. And we're working very closely with community based organizations that already have that connection with the community. 

Early wireframes of design evolution based on feedback. Each iteration takes a few weeks to execute.

‍

Example of listing page evolution of a listing page.

Long Term Roadmap

Sethna: 

Thanks, Wendy. So, you know, just highlighting, you know, maybe even just going back to this list of problems like we mentioned the PRD, which basically stands for the product requirements document, that was what the Google Civic Bridge team put out. This was included there. Being able to really define the problems people are having and in searching and applying for affordable housing and all those problems coming out of the research that Wendy mentioned interviewing people, observing folks we were really grounded in what the actual needs were, and that is what kind of informed our long term roadmap. This is just a high level roadmap. The journey is at the top, you know being able to see a listing apply for it. Lease it up. That's a very, very high level journey. But throughout the past five years, we've built out different pieces of this and, you know, we would zoom out and say, all right, we need to get listings right. 

First, you know, we need to have the right information for people to browse and see what they're eligible for. Then we can kind of move into the application process. Then we can build tools for people to manage incoming applications. So we were kind of able to put these different chunks of work around, along wrong term, along a long term roadmap and continue to do that today. You know, there's stuff that was built out in the last two years. There's a continued roadmap for the next few years for DAHLIA. So it's an active product that still gets managed and supported. 

Get Started

• How To Conduct User Experience Research Like A Professional

• User testing: the basics

• Research and User Testing Primer

• Tips for Running A Design Thinking Workshop with Your Team

• What is a Customer Journey Map?

Tools & Templates  

• How to Create an Effective Customer Journey Map

• Customer Journey Map Template

Larson: 

Thank you all for the in-depth presentation and look at DAHLIA and its impact. So if folks are interested in taking the next step or applying some of these learnings in their own work if you're interested in this process or user needs Jane Lim of our office of civic innovation is available to connect with about our Civic Bridge program. As well as Mai-Ling Garcia who is here in the audience is here to support digital services and how they can help monetize your services. So really if this is a space that you're interested in there's resources and folks in the city that are really excited to work with you and hopefully you know, apply some of these learnings. And I think with that, we can kind of move over into the Q and A portion. 

Q & A

Seeing as this is a new transformative system, new service delivery, what's the adoption been like for city staff? Where have there been challenges with implementing this new digital system and how have you been able to overcome them?

Roeder: 

Yeah, well, that's a great question, Mat. I guess what I'll say quickly is that Jay Nath, who was the head of the mayor's office of civic innovation at the time suggested that, and in fact hooked me up with a good culture shift book and I was all ready to go thinking, okay, I'm going to have to sort of help my colleagues adapt to this new reality. And, and I had been surprised just how, how much we've embraced it. And I think it's because we get just how much it's helpful for our applicants, but also the fact that we kept asking them along the step of the way, how does this change your business process? And if it doesn't again, if it isn't useful for every step of the way then, then we'll have issues. So it's been surprisingly easy for us, I have to say. 

Roshen and Wendy, have you two had similar experiences with other cities or municipalities that you've worked with? Are there any other techniques that you've seen that have been helpful in this adoption and continued maintenance of these new systems?

Sethna: 

It's just about understanding what the existing processes and systems are, whether it's a spreadsheet or something in person or a paper application. Throwing digital at everything doesn’t always make sense. So understanding the pieces of the process, for example, this is an application to be in a lottery, then people get ranked and selected. If someone is coming in that has gotten a unit, all of that process is still done offline and in-person because it requires so much kind of back and forth and paperwork. So we haven't particularly digitized that. Or if, the spreadsheets the city staff were using to lease up the building and report back and forth, we really understood what went into those spreadsheets and figured out this part actually could be made easier in a digital realm. It’s about understanding people's current processes and how we could help make them better. Rather than saying, we're going to drop this new tool on you and you have to use it and, you know, just figure it out. 

Fong: 

The only thing I would add is the investment that we put into the relationship building from day one was essential to help the internal champions like Barry. We, for me, I felt very invested in the product and, and the team. So the success was very personal to me and yeah, I think that was a huge part. Just having that collaboration across all the teams. 

How do you streamline income verification that once required pay stubs?

Sethna: 

So again, the application that we developed online was an application to a lottery. So what we realized is that at that point, there might be hundreds or thousands of folks applying, and we only need to collect a baseline level of information. So we asked people in that online application for their income, they have to fill it in. And you know, I showed you guys that the, the system says, hey, you have to be over 18. It would check income against eligibility of the listing. So it would say, hey, it looks like your income is too high for this listing or too low. Just so you know, like you're probably not eligible now. Someone could change that, their answer and keep going through the application process because actually with fair housing laws, we have to allow everyone to apply if they want to. But that was one way we checked it very lightly in the beginning. And then after someone is selected and, you know, they get a unit, then we do a more thorough income eligibility check. Then the leasing agent kind of goes through their documentation and understands. 

We found that that early later check helped get more eligible applicants through the system. And we didn't need to ask for all the documentation, all of that upfront cause it's also people's situations change and a lighter touch upfront was more useful. Barry, if you wanna add to that.

Roeder:

For the folks that were selected by the lottery process, meaning that they did well enough to have a subsequent interview with the leasing agent or property manager. And, and by the way, at one point we thought we would build a digital process for that. But what we heard was no, no, no, no, no. There are a few enough people and we like to sit down with them and it's a better experience for everyone. We thought, great let's do that. But in that experience, they would be asked to bring in whatever supplemental documentation, documentation, including pay stubs and things like that. But that's the point, which is okay, if you come in with these papers and you are, and everything checks, I'm going to hand you keys to a unit when you walk out of here. So it's worth people's time to sort of put that effort in to get all that together. 

Can someone speak to the challenges of the effective date of new changing legislation and the ramp up time to build new tech and solution?

Roeder: 

I'll try that one. That's that's what, that's a tough one. I think what I would say to that, Glen, the more time you have, the more advanced time you have before something takes effect the better, because it gives you a chance to prepare for things. But one of the benefits of this agile approach, if you will you know, again, the idea of, okay, well, let's start with something is it'll give you an opportunity to sort of set and say, okay, let's, let's imagine that you find out today that there's going to be some, some, you know, significant change in your legislative landscape that happens in three months time. And it's like, Oh my gosh. Okay. All right, well, what are the core pieces? Is there adding the ability to sort of negotiate with that deadline? If they say, no, no, no, listen, you either have to shut the lights off or do it like this. And then within those parameters say, okay, what's sort of the most productive way that we can focus our efforts on those core bits. 

There's a phrase that was used and you may have heard this in this context called MVP. And my joke that person's heard me say a thousand times is when I first heard this, I thought, why did somebody start talking about baseball? But they mean the minimum viable product. Like what's like the little, the little, the tiniest little morsel that we can get out there and get things started. And hopefully there's a way to find a way that the minimum viable product can at least meet your legislative needs or at least placate those things while you're in the process of getting things more properly ramped up. Hopefully that answers a question and I'd be happy by the way. We'll, there'll be a slide or if there wasn't, we'll bring it back up. I, I saw it. I'm going to go welcome to reach out to any of us, please, and, and follow up with further questions. Be happy to go into this integrated table, if you like. 

Sethna: 

Sometimes our current processes aren't meeting policy requirements. Right. And we have to recognize that like you know, in the San Francisco policy of inclusionary housing, that any new development needs to have a percentage of it rented out below market rate. A lot of those developers are market rate developers. They're not really sure how to, you know, find an eligible applicant and things like that. So you can even recognize gaps in the current process and say, Hey, let's start there and like resolve that pain point so that you know, everyone can actually follow the policy and the legislation that intends to. 

Have you had to plug into any other city systems at different steps of the process and if so, how have you navigated that work? 

Roeder: 

The answer is absolutely yes. When I see savings systems, I think from a technology perspective but they're also city processes, but let's set those aside for a second, just talking about city systems. So most of you had made a prior decision to invest in Salesforce and there's some benefits to that. You know, like most things, Salesforce has some, some pros and cons, but there's a robust sort of permissions and security architecture that we like and we thought, okay, great. So Salesforce is the place where all the data is stored. The web app that Roshen walked through for you really only holds information sort of account log in and some basic things like that, but it's API, application program interface. It's basically sort of talking back and forth to Salesforce, to store data or get data as required to get through an application or whatever process is involved. There had to be some modification of the design to make sure that it worked with Salesforce. 

We have some housing lottery preferences which basically give you a leg up. It sort of gives you a boost if you have that. And one of them is called the “live in the neighborhood preference.” And so DAHLIA does a really cool thing where first of all, it says, okay, you just typed in your address, let me go out to this other non-city service and say, can you tell me what's the right way to say this address? And it comes back to the applicant saying, did you mean “123 Market Street,” as opposed to maybe they typed in “123_Market Street.” Then it takes that information and it runs it through a DT, department of technology, server that says, wait a minute, can you tell me, does this address that this applicant put in, does it fit within, what's called a polygon, basically a map. Does that fit within the map of addresses that would be allowed this special live neighborhood preference? And it does all that fairly instantly and comes back and says, if things are good, hey, congrats, you qualify for the live in neighborhood preference. So you're right. There are definitely places in which we had to intersect with other bits of technology and make sure that everything fit. 

A final comment again, on that process that I set to the side, there are sort of official processes that we either needed to follow, or we needed to respectfully request that we make some modifications to. So yes, there's a, there's a process involved in all of this, but hopefully again, we pair this back to just sort of what are the minimum requirements so that we can get something started here and we'll build on it from there. Roshen or Wendy, would you want to add anything to that? 

Sethna: 

We built the web app, which is basically the website that I showed in a really flexible technology that allowed us to interact with Salesforce and the other tools that Barry mentioned. Again, the housing listings, the information about the actual listing, as well as the applications people would submit. All of that was sent back to Salesforce and stored in Salesforce or pulled from Salesforce. We didn't store any data or information on the website. 

Technology choice is an interesting thing just to understand, you know, what is a sustainable technologies choice for a department? That's something I think that is definitely good for everyone to coordinate through SF digital services and things like that. They could definitely guide you. But making more modern technology choices kind of helped us one deliver a really good user experience because in those technologies you have more flexibility to design for the user. But then also using an API to the Salesforce database. Making those technology choices up front. The other thing I'll say about technology choices, just understanding how you can support it in the long run and what kind of engineers or support you can get for that piece of technology in the line. 

For anyone who's interested in moving their services from paper to digital. What piece of advice would you give them?

Sethna: 

Sometimes we get a little caught up with that. The first mistake I think I see a lot of city and County kind of government teams make is like, we know what we need. We need a mobile app to do this thing, or we need to build this to do this thing. And I think it is really easy for all of us that have been in the work for decades to say like, yeah, we know what housing applicants need or we, you know that's kind of the biggest mistake I see. So I think flipping that a little bit, just to start with, instead of saying, we know what we need, we just need to put this form online, instead of saying that, flipping it to doing the interviews, to understanding what you don't know yet about the process. And then really just starting as we say in the, you know, design and tech industry, low fidelity. It may be just get something out on a piece of paper or a whiteboard and you put it in front of that user and say, Hey, like do this task and see if you can do this with this little prototype that I made. So starting really simply and going back to the fact that maybe we have a hypothesis of what we need, but we don't know exactly what we need yet. 

Fong: 

I would also say start small and rather than jump to a solution fully. Queue into what you're hearing from your community, like, what are those basic needs that are not being met? And it may not be a technology answer. But what else can we do to get more people on board? Maybe I'll just set up a workshop internally just to get some ideas to get started and start small. It doesn't need to be this giant project. We can just start by identifying those key needs and how we might support our community. I would just say that and look for internal champions. 

Roeder: 

It strikes me as you guys talk, I remember the points in time. Like last week. I attach myself to certain outcomes. I get excited about, ooh, I think now we're going to do this. And they, again, I might've done them in a vacuum. Like it's me imagining what will happen next. And so part of it, I have to challenge myself as, and we as mostly the department to challenge ourselves to, well, let's be open to what this needs to look like. And sometimes that may feel like, oh God, just do we have to have another focus group? Or do we have to do this again? But there are some really rich benefits that come from that including might be, oh my gosh, like I mentioned before, we don't have to do a digital long form application at all. Oh, that's good that we learn that. So I think being open to that, it's great. 

The last thing that I would say is just that you don't have to do it alone. I mean, there's such fantastic resources for this in the form of the Office of Civic Innovation in the form of digital services, in the form of other people, other departments that have done this in the form of great resources like exigent or things like Code for America that some of you may have heard of to really sort of say, okay, I haven't done this before. Can you help me sort of think this out? 

One final comment is I loved sort of the initial title of this conversation, which was sort of digital. It made me think of like, well, just put a paper app in the scanner and make it a PDF and you've digitized your paper out. But as you realize, I think from this conversation, and there's so much more involved. It's been a rich experience for me personally, I think for the department, we do things differently now as a department, as a result of this. I'm excited for all of you that are interested in this and, and wish you the best in your process. 

Thanks, Everyone!