AI disruption in e-commerce – David Drollette (Wayfair)

[Applause] I've been away for about 12 years I run the data science and business intelligence teams we've got some great support here I see some of our data scientists in the front row thanks guys for coming so very briefly before I get into the details of the talk I do want to take a quick second to briefly explain what Wayfair is I'm sure there are a lot of people in here who actually haven't heard of us before so wafer is a tech company that just happens to play in the home furnishing space you know hopefully you've at least seen our commercials the jingles are very catchy they're definitely ear worms so if you haven't you'll see them soon and then this is about as simply as I as I can explain our mission so we want to empower everyone regardless of income level to achieve a home that they love and I'll focus specifically on visual searches at case study but there are a lot of other efforts going on in the data science space at Wayfarer and feel free to ask me about them later I'm happy to chat about them so what we're gonna do is focus on the evolution of search and how that brings us to the Joule search so over the last 30 years I think we've seen a pretty big explosion of advances and ways to find the products that you're looking for obviously owing very heavily to the advancement of technology itself so like I said we'll focus on disruption in Wayfarer space but let's first talk real quickly about how we got there Dee chose to start his talk in 2017 I'm gonna start my talk in 1895 I think it's important to recognize that the catalog actually was a pretty big innovation at the time you know it essentially brought the store to you you can see I have a time to value scale that will be following over the other disruptions I'm not making the argument that catalog is AI by any means but it absolutely can bring the store to you and really opened up selection for a pretty big part of the country it was not at all my intention to select companies that are no longer doing business that was sort of an accident there is a company up here that's still in operation so East Bay was actually acquired by Woolworths and is now doing business under the name footlocker so they're not all gone now let's get into the AI side so these are great screen grabs from almost 20 years ago you can see product design has come a long way as well but you know let's talk about digital search so you know you think about Amazon you think about eBay even 20 years ago the catalogues are far too massive for your traditional sort of print catalog approach and so both independently developed artificially intelligent search systems which made finding the products in these massive catalogues keyword easy so you can see we're continuing to move along the time to value scale things are getting a little easier for the consumer we're finding things faster so let's keep moving down that scale you know taking this search technology to the next step we have you know any internet aggregators we've used Google here you know they use search technology to consolidate the Wayfair fee the Amazon feed and everyone else who's willing to provide that product feed to Google which is a big leap forward for the consumer you know especially when you think about commoditized spaces I've used Keurig here you know any number of a thousand examples where you know there's 30 40 50 retailers trying to sell you the same thing how do I make sense of all this in this search aggregation is actually a really great way to do that Google Shopping used to be called frugal the link is still active if you want to check it out it used to be free for advertisers it was kind of a bummer when they started making us pay for it and making everyone else pay for it but the value is absolutely there an absolutely great example of AI disruption continuing to move along the time to value scale we're getting deeper into AI you know getting into personalization and recommender systems you know technology essentially becomes your personal shopper in examples like these I've used stitched fix I know they have an IPO coming which is very exciting for them but essentially we can take all of your data you know whether it's X data when you're actually telling me I love this thing or whether it's implicit data you know we can take your clicks we can take your Pinterest saves we can take your views and we can translate all of that into helping you find what you're looking for I know it Wayfair we tend to hop back and forth between the term search and the term find because really what we're all about is product discovery how do we help you get to the product you're after as quickly and seamlessly as possible and again we mostly use actually implicit signals today but recommendation systems again another great disruption for for e-commerce just helping the customer find what they need even faster and then a big leap forward on the time to value scale new instruments of search so we've chosen the voice-activated devices Amazon Alexa Google home this is really upended at least the way we shop if you think about a voice assistant that's capable of crawling you know the massive Amazon catalog it makes things like reordering a consumable something like laundry detergent if you don't have a – button or olive oil or something like that or making an impulse purchase that much easier that much faster and what's really incredible about these is it's enabled us to do that while taking us away from what we traditionally consider our shopping devices so you don't have to be at a laptop you don't have to be in your tablet you don't have to be on your phone and you can still just shout across the room and say hey Alexa I'm out of Lay's potato chips you know let's let's get that refilled so what's really interesting also about voice search is it actually leaves itself open to disruption I don't know how many folks here have heard of the Google home of the whopper campaign that Burger King put forward so Burger King actually released a TV commercial that could activate your Google home and got Google to start reading you the Wikipedia entry for the whopper you know Google figured it out they put a stop in with Burger King released a new ad that did the same thing so internally enough you know as you're disrupting at Lightspeed you do leave yourself open for just in and of itself which is very cool I urge you to check it out so really meet a really clever campaign Burger King and then we're getting to visual search so I think one of the big things that I want to point out for the rest of the wayfarer slides that we go through is our ability to quickly translate ideas for customer experience in the supporting algorithms to features in production is really pretty key for our ability to disrupt in the AI space in general so for style oriented verticals like home decor you know visual search visual shopping has a real opportunity to disrupt you know if you think about search in general it's really easy to hop onto a site and say I need a four pack of Duracell double-a batteries you're gonna find what you need but what if I were to say help me find that really great industrial metal bar stool with a square top and cross support legs try to type that into Google and see what you get I think visual search really helps helps again increase the time to value you can see I've actually left some time on the right of the time value scale for a couple reasons first I'm not so arrogant to think that there won't be further disruption that continues to move us further along that path and then second in e-commerce I don't think we should forget about logistics we're always going to need to bring the product to the customer so there's always going to be a sunk cost of transporting the physical good from A to B so let's get individual search how did we do it visual search was actually a hackathon project at Wayfarer which is very neat so we took four incredibly engaged data scientists and engineers and they spent the weekend building a prototype the prototype is called the cornet you actually have real shots from de cornet that was built in two days your initial results you can see weren't great on the far left you have the user input so we took a snap of a tufted couch we got back actually some tufted couches it's pretty good we also got a chair in there not great but this is two days worth of work so I think the big takeaways from the hackathon project we're really one creates space for your all-stars to innovate like we said we gave a really passionate team you know a weekend free compute resources and this was they came up with was amazing then I think number two is build the MVP and end prove your viability if you're trying to make a case that this should be done it helps to show the user what its gonna look like this actually won the hackathon and our judges are essentially all of our c-suite and they're really blown away because they could take it from search to result and really see how this would impact our product and then the other thing that's really neat to mention is Wayfair next which is Wayfarers 3d and augmented reality program was also a hackathon project so again this model really works and you can do some really incredible things with a small amount of people in a pretty short amount of time so what do we do once we think it's going to work we put a team around it so what you have here is actually the team as it stands today working on visual search so we obviously have product managers right there quarterbacking the team they're providing the overall vision and they're keeping us on task right there excellent project managers and they're keeping us moving we have data scientists we have search technologists we have business intelligence analysts so what are we doing there we're obviously developing the underlying deep learning models we're working very closely with the next group to store for an engineers to put the product into production and then we're also focusing on algorithm SLA right when we're talking about search you don't have five seconds so you don't have ten seconds you have 100 milliseconds you have 200 milliseconds to get this search executed so we need to keep that in mind as we're developing the algorithms we have store for an engineer's they're connecting the algorithm to our users on-site and they're actually deploying and maintaining the digital search model and then we have creative right you know we couldn't design the experience with our creative and UI UX folks so they work with product they work with us to understand what we're really trying to do for the customer and then they developed user experience and these was needed on-site creative so you can see all told you know we have about 12 to 13 people working on this it's been going pretty well the next slide I think a lot of people like this is nuts and bolts as I'm gonna get today so this is basically a diagram of what we've put together so on the technical level our investment was one establishing the appropriate modeling pipeline and then to actually physically acquiring GPU resources to make this all work Wayfair is not in the cloud so we do have on-prem data centers so the actual hardware was a very important part of this whole process so just working quickly from left to right you know we have our filename pairs with labels I think that's one of the things that makes us so successful anyone in this room will probably tell you you know an amazing data scientist with a teeny tiny amount of data is not going to beat an okay data scientist with an exceptional amount of data I think that Wayfair probably has one of the best if not the best labeled training sets of furniture data I think that's what's made us successful so on the far left you have that training set the training data it's labeled pairs of images essentially saying you're a match or you're not a match so we have workers pre-processing the image files turning them into numpy arrays then we push those into a shared queue where the training system can actually grab those grab those images on a separate set of threads and you can see we have compute decoupled from storage so we can continue writing back you know model checkpoints training statistics training loss validation loss and that's really important I think for two main reasons one if this thing crashes we don't have to start from zero we don't have to retry the model and I think two we can jump out we can implement early stopping at any point if we start to see overfit happening you know that's obviously a challenge as well and for those of you I actually will get a little deeper into some details here on how we actually are doing this thing so the learning algorithm itself is stochastic gradient descent and we're actually using inception v3 which is convolutional network from Google our learning rate schedule is linear and we're using transfer learning so we actually took inception b3d that was trained on the image net and then we applied transfer learning on our own data set to slowly transition this model to our task and then for tools at our very high level we're using tensor flow with Charis Charis allows us to move a little bit faster and then finally on the hardware side we're using the latest and greatest in GPUs so we're on nvidia tesla P 100 GPUs which is the latest and greatest for now I know a volta is coming soon but we do have some pretty serious horsepower behind this thing so how's it doing so there's really two primary KPIs when you think about the performance a visual search there's how's the model doing empirically and then how are the customers responding to it right you can't have one without the other if you have the best model in the world the customers don't love it that doesn't really it's not gonna get you anywhere so on the left our primary success metric for the model we use recall at K which basically means of the set K did I return the image the exact match of the image that I'm looking for so we have a validation set that the neural network never sees and that's what we're feeding in to establish our recall numbers and you can see we've been able to make pretty good step function improvements from the image net baseline through our first training set you know to our next big set of architectural changes and then on the right we have the product side KPIs you know what are people that are doing what are people that what are people doing that are using this feature you know how are they reacting how are they responding you can see we've used a pretty high level KPI and this is just seven day return rate after you've used visual search the inflection point in the middle it's actually when we rolled out v2 of the model that blue line you know I think a lot of folks are gonna ask you no big deal why are you using seven day visit rate it's still a new feature I think one of the biggest things we're really pushing now is customer adoption we know it's great a lot of people don't even know it's out a lot of people don't even know that Amazon has a visual search feature so customer adoption is very vague and as we get more and more data into the system will move away from something like a repeat visit rate to more lower level KPIs something like you know Add to Cart or purchase or something like that and then I know that the title of the talk was both the present in the promise so this is the promise aspect of it what's been really amazing is that this hackathon project the visual search has spawned a huge number of threads after we've demonstrated the success of what this thing can really do so I've got a few examples here there's actually more than this that we're going into but things like implicit product recommendations you know that's generating recommendations for you based on what you're clicking looks like instead of using something like a collaborative filter we're actually using the images to German generate products that have similar styles and I'll show you slide on that merchandising you know how do we find duplicate products in the catalog it's a terrible customer experience when you see five of the same thing right in your face on the page so we've been able to apply our visual embedding to that problem you know being able to identify all furniture within a room all at once is actually my favorite one it's the last slide I'll show you that at the end and then marketing you know getting into how do you predict whether a customer is gonna react favorably to an image at all it's a really big deal for the creative that you're putting out there for the customers and so we're starting to get into modeling the quality of an image itself to see if customers are going to react to it so real quick I'll go into some great examples here I mentioned visual recommendations this is a very quick how we're doing it so the Browse context is what the customer is actually looking at you know what are they clicking on we feed that into our visual visual embedding space and we essentially find the products which are closest to those products that are being clicked on you can see anecdotally it actually looks pretty good the styles are consistent and we're able to do this across class which is something that collaborative filtering and other recommendation systems have actually struggled with using for exam dresser data to recommend a bar stool we haven't seen collaborative filters be successful there but our visual embeddings you can see anecdotally looks a lot better interesting finds I'll give sunandha a shout out this is one a perfect wonder for big things I personally love this effort because what we're doing is we're establishing a space the example here is coffee and we're saying great find me things that are within this space but are visually different from one another and you can see that really coming through and the interesting finds results page it's a pretty compelling page it draws your eyes to it and so we're grabbing things we're essentially grabbing the purple dots on the outer ring of the space knowing that they're reasonably far apart in the visual embedded space but they are in the same in the same theme and then like I mentioned my absolute favorite identifying all the products in a room so that you can shop them so Wayfarer works with thousands upon thousands of interior designers interior decorators and they submit amazing photos like these to us being able to go through the exercise of making this image live in shoppable in a matter of seconds instead of a matter of days or weeks you know using offshore resources much for tagging is really exciting and I think really compelling again from this home decor space where it's all about visual inspiration visual shopping you know I personally find it a lot easier to shop something like this rather than you know a grid of sofas and trying to find what I really like instead you can start with a scene you know find a scene you really like and then pick and choose from there you know what's really interesting about that scene so that's all I brought with me I will also say that we are hiring we are always hiring please feel free to find me find our booth out there actually play with visual search we have it set up live in the booth you know it can be fun to also take a selfie with visual search and see what piece of furniture you most resemble a recruiter Michelle try this right before we got started her second result was a picture of Jesus which was very interesting but thanks for listening folks [Applause] [Applause]

2 thoughts on “AI disruption in e-commerce – David Drollette (Wayfair)

  1. Dead on! ecommerce & the "mom and pop" online stores will be the most affected by AI if they don't adopt fast!

Leave a Reply

Your email address will not be published. Required fields are marked *