(462) E91: Investor Tomasz Tunguz on AI’s Breakout Year and What Comes Next - YouTube https://www.youtube.com/watch?v=fHPJaulSCd4

Transcript: (00:00) hello everybody Welcome to the internet and to this podcast it is episode 91 of Topline I'm joined by my co-host oid Zaman and AJ Bruno CEOs respectively of sales talent agency and quotapath but we've got a very special guest in front of the pod on tamas Tang General partner founder of theory Ventures uh formerly redo Ventures one of the foremost prognosticators on SAS and artificial intelligence and crypto the three most important topics in the modern technology ology o I don't know is that a word Sam o u v r I never (00:36) know how to pronounce that word but it's you know like the overall course I would say welcome to Topline the podcast for the best Founders operators and investors in B2B Tech every week AJ Bruno AA Zaman and me Sam Jacobs will break down what's important for you to know to be the most well-informed professional in the market your deal is almost closed and all that's left is the security review but when it comes to those lengthy questionnaires the endless back and forth can often cause deals to stall out leaving your deal at risk and dollars on (01:22) the table with Fanta questionnaire automation go to market teams can complete Security reviews up to five times faster helping you close deals and less time than ever visit v.com toline to learn more about questionnaire automation toas welcome back thanks for joining us gr to be here happy New Year to you all happy New Year's to you Austin's gonna kick off the docket but before we get there you have recently said I'm hijacking this conversation because we're all very curious you've said that you use AI a hundred times an (01:58) hour is that true and explain that to us and how are you using second this very are is this really you that's important yes so I use dictation a lot so big chunk of that 100 and so um there's a button on my keyboard that I can pull down and it will call up a an AI that will listen to my voice and it will take the audio transcribe it into text and then that's passed to another AI that then figures out what to do with it whether like format it in a list or cre a paragraph for eliminate the UNS and so that's a big part of it there's another AI that I (02:36) use a lot as you can imagine we research startups where I can hit another button on the keyboard and it will summarize a website or summarize a research paper uh so if we're ree right now we're researching something called text diffusion models and whether or not that's a bindable architecture H the other thing that we do is we transcribe podcasts and we extract the text from them and so the I would say like a big chunk of it is both on the dictation and research and I don't know if you guys have played with Gemini deep research at all but (03:09) that's that's a pretty special Pro and um and then when whenever we write or publish we're we're using I think Claude in particular is quite strong at um capturing a v sword I really like to use it this might be a little masochistic but I had this amazing AP English teacher his name was Mr Dunn he was a former military gu I didn't like to write until I met Mr Dunn and I didn't like to write until maybe the second half of his class because he was just he just held a really high standard and now I asked my AI to grade my blog post like (03:45) Mr what I love that you you used a math teacher to love now that's you you like to write because of a math teacher that's pretty impressive by itself oh he was an English teacher did I say oh he was English did you say math he English I heard math where where my head is today are you uh so these models are are they running out of your system directly so you've it's not the traditional chat GPD stuff you've actually buil installed some on your computer and you're using those to then feed into these other models yeah (04:21) you can you can use there off the shell tools that will use CLA and open Ai and others I think I want to see them work I want to understand what's breaking what's not working what is the application developer building around the AI and so there's one technology called AMA that will allow you to run models locally there's another technology that's actually part of AMA called Forex doso that will allow you to run local models and what you'll find is some of the more some of the smaller open source models like Gemma 2 (04:54) if you want to do any kind of text editing it's a two billion perimeter model which is small got two billion then you have 9 billion and you have like 25 billion they go all the way up to 475 today um but the small models actually work work really well for basic text editing and you can think I think about them as basically like encyclopedias almost with brains I don't need a big encyclopedia to edit text like a very small one will work and then you can use something like L 3. (05:20) 1 which has the same performance as gp4 but you can run it locally on a pretty powerful KN um and then I've even tried this the most recent LL 3 which is a beefy 75 billion parameter model and that will give you like pretty close to Claud performance it's slow um and like I said I have a pretty BC machine but it's pretty slow so you have to be patient but you can get it to work and then you can see what's coming off like where where is it making mistakes and I posted a video yesterday uh playing around with an agent that was booking a (05:52) flight and that was quite intense I saw that I yeah it's amazing the on I mean like the way that the baling box are created around the Google flights and you can see like it makes Contex for the audience if they haven't read it um he basically installed an agent that went and booked your flight for you very simply like you just triggered it I'm looking for this type of a flight it went it looked at the various flights it navigated around the popups and everything else and came back to you with information that you were able to (06:23) act on relatively quickly right yeah it took it took less than it cost less than three cents one reader asked me how much did it cost I went back and looked it cost less than three cents I think it's probably a fraction of a a scent because I tried it a few times and then um he was able to find the two cheapest flights from San Francisco to New York for the certain dates actually there was a very astute reader who noticed that the the AI made a mistake which is it broke two flights from San Francisco to New York so if I it wasn't perfect it (06:54) was not perfect but the I mean we'll get there the cool part is you can see as you watch the computer you can see how it creates boxes around this is where I type in EWR and this is where I type in San Francisco and this is where I figure the address and then if you've been on Google flights or any sort of online travel agency at the bottom they're all these SEO optimized links it says like Las Vegas flights and Phoenix flights and it makes a mistake and it clicks on one of those links and goes to New York flights you can watch in the logs if you (07:23) look on the logs it says made a mistake going back and so it actually clicks the back button goes back and then starts to flow again and so I think one of the reasons I like running these models locally and tested them is you can see that happening in a way that and it's wonderful that people are building off the shelves and much easier but I wouldn't I wouldn't have had that Insight just in using somebody else's tool for example are you constantly changing models or thinking about like every is this a week toe change that (07:52) you're making you rattled off about 10 different things is that is that a list that you just continue to iterate off of yeah I think um I think it's okay so we were we were at this event and uh there was a leader of one of these foundational model companies and he um you know he he pulled the audience he said how how many of you think that AI has improved a 100 times in the last three years 100 to a thousand times last three years all this raised our hands and then he asked how many of you believe that AI will improve 100 to a (08:27) thousand times in the next three years and most of us rais our hands and then he said well if you believe that you should be living completely differently interesting and yeah I left that meeting I was beered he said that because you're all investors and you should be on the Forefront of new technology and you should be doing what you're doing that's what he meant he didn't mean as sort of uh regular Average Joe's you know lay people yeah no no I think I think he means both I I think I think he means both and so I (09:04) left that meeting bewildered because you know I was asking myself how do I how do I start to live in the future if I don't know what that future looks like I have no idea right like if AI is a thousand times better and we have agents that PhD equivalent are better literally every domain at her fingertips like what you know what wa who go and so like I and so it made me wonder like well that means that every single process that we do at work is likely to be completely changed if we are working on a computer is there anything that's (09:45) scared you when you've gone through this you're like whoa like either you weren't expecting I mean you mentioned the back button but like just that were like that you feel like you're you are now living in the future and I don't know the unknown can cause a lot of fear and panic but it doesn't seem like in that way I think I mean I think there's this shell shock that oh my go this is happening really fast and I better start running because if I don't start running I will be outdated very quickly I think that's extremely natural (10:16) human emotion and I had that when you made that blog post about how much you were using it that was that moment for me where I was like one second like there's I thought I was using it a lot but this is a difference by many factors and so I had to really like sit down and rethink how I was using AI because it was nowhere close to like what your usage was like not like it wouldn't even be in the same realm and so you do have those moments and since then like I've adjusted it I'm using it a lot more and the more you use it the more you (10:49) understand how to use it and when not to use it as well like where it's not working that well yeah I I think um I mean like writing Cod uh or whatever I put together there's this software language I know nothing about called Swift I put together an app in Swift and that poers one of these applications where I can hit a button on the keyboard and that couldn't tell you what it does I mean I can read the comments and I know when it hits the button it does what it's supposed to but it couldn't tell you like could could not reimplement it if (11:21) you gave me a swift interview as part of an engineering battery no how do you quantify the difference between let's say product I I think productivity ends up being this like element where all of the effects come down to net productivity of a person right I can do more and I can do it better in the same amount of time have you been able to quantify the difference prei and not just pre- like chat but like back in the day versus today you using it as it is today with its strengths and weaknesses what's the actual difference in your (11:57) abilities productivity Etc yeah I so I haven't Quantified it but I can tell you there's a difference in feeling so uh and I don't think this is a new skill right I I I wrote a post a long time ago was talking about the missing class from B school from business school which is delegation I went to graduate schools went to a good business school and we were not we were taught to negotiate we were taught operations research we were taught accounting all this another thing we not taught how to delegate and you you were all leaders of companies and (12:28) I'm sure you all Ive delegators what I've noticed with the AI is if we bring a mentality of Delegation of writing a long prompt giving it a long context just the way that you might have a project brief that you give to one of your departmental leaders it works really well and so now the question with AI I find is I can stop at this project scope I can write the PRD and then I can ask Google deep research go on Research text Fusion models and here are the six questions that I really care about then I can have two or three followup questions that (13:04) used to be that used to be someone right who would go and probably me candidly because we're a small shop it used to be me and I would go and spend a week or or three or four days going and reading the research papers highlighting putting it into a document and then summarizing and now I can just type in in the Google deep research and five minutes later I have a two and a half page memo with all the links to the papers and then the next thing is this paper is interesting send it to my Kindle right so that I can (13:32) meet it or send it to the team that level of automation hasn't happened yet but this idea like the number of things I delegate to a computer is much higher than it was you know I know we're gonna talk about voice chat but one thing you just said that reminded me is like there's so many times where I'm actually going back and forth where I'm like can you just email this to me and I just want to check to see if it eventually gets the capability to do that eventually it's going to say yeah I can do that but it's still not there (13:59) unfortunately it wasn't one of the 12 Days of Christmas that uh open AI set forth unfortunately not there yet the one the one I was happiest with the Gemini I have an Android phone and Gemini finally launches Spotify integration that's when now you can say play whatever song you want and it will play which is pretty cool um yeah the email one or set a reminder those kinds of things would be awesome do you have a do you feel like you're obligated to have like a a middle to long-term view on the end state for human productivity (14:30) or for Humanity you know be I it's like there's these two levels that you operate at one of them is the trains leaving the station I've got to stay up to dat I've got to incorporate these tools into my daily workflow because I don't know how the world is going to change but I don't want to be one of the people that it changes past when I can't catch up and then there's a that and that's that can be a fairly tactical lens and that's you know it's sort of like the The Singularity like you're very zoomed in on the slope of (15:00) the curve and so you're like I'm going to change this thing I'm going to do this thing but then meanwhile if you zoom out there's a a world that we're moving towards and that's a question that I don't actually know I don't know what that world is that we're moving towards is it a world of overall the productivity gains Drive massive economic output in some way and so the overall global economy is great and all of us are basically like micro influencers or we we've got like these agents we've got robots at home so we (15:30) don't do any chores and we have all of these AI agents that are doing all of our boring tasks and we're just having sort of fun playtime a lot like what do you have a point of view on what the world will look like yeah yes you know it's very risky to make these kinds of predictions very hard I think I mean you know what was the first robot people's homes vacuum cleaner microwave washing machine yeah it was a dishwasher yeah and and now like I don't know about you guys like my favorite kind of a pet is a Roomba because it cleans up after (16:08) itself right I don't have to feed it feeds itself off the the crumbs that we leave on the kitchen floor so I mean we have lots of robots in our house and yet I still feel like there we have the washing machine and the dryer those are all automated to some extent and becoming smarter and smarter but still they're still chores um and so I think it's the same in work like I don't think we end end up in this sort of um oh man uh like a Lotus eeding world where we're just like hanging around and um s just like Sam really wants Wall-E to be a (16:47) thing go live in a ship and sit in a chair but I do think I think there's a massive productivity wave coming in service work massive massive massive massive productivity weight and like we can talk about like the public markets and analysts will tell you that equities are massively are at 20-year highs relative to bonds because of the implied premium between those two asset classes and I understand that and the Optimus me wants to say well yeah but reality is like a bdr will not manage one inbox of 40 emails to manage 50 inboxes of 40 (17:24) emails and um all you know other people inside of like the will probably double or triple their productivity because instead of spending 75% of their time in meetings and filling out Salesforce they'll spend 50% of their time or 60% of the time in front of customers so I I think there's a pretty significant productivity gain and in the next like call it three to 10 years that productivity gain will not be evenly distributed and when that happens you have big shifts in market share just very exciting for Venture capitalists I (17:56) think should be very exciting for any kind of investor any kind of Innovator because the status quo shakes out and then at some point everyone has it and we get back to a place where we start adding humans to companies again in large scale because there are these processes that we find that are not tractable they're not problems that AI can solve and so now we need more sophisticated brains to go and solve them again what are your thoughts on the speed at which this happens so with every platform shift the speed at which (18:28) it becomes mainstream seems to be increasing like if you look from the internet to mobile Cloud Etc and AI if you look at where it is today it's gotten there much faster than previous platform shifts had gotten but at the same time you'll hear people say something like Cloud ad option is still 30% of workloads and it takes a while and when I talk to companies outside of tech great companies companies publicly some are publicly traded well-known names when you think about that Tech stack and the sophistication of the (18:59) tools that they're using it's still it it's Leaps and Bounds behind where your Progressive tech companies will be right like I I like to give the example that if you ask them what's your TCH stat for your sales team you hear CRM and sales Navigator you know you tell them what about gong and they're like what the hell is that like and so what do you think is the pace at which this becomes implemented because it's obviously moving faster and it's really disruptive and it offers advant ages that are not incremental and that are significant and (19:31) that should make it you know a little bit faster than um before but yeah what do you think like three years in do you see more adoption of AI than we have Cloud adoption today do you think about it like that yeah I mean I think you know well like you said cloud is 30 to 40% of workloads today it's like one and a half trillion um and that's taken call it 24 years to get there so I think AI adoption to reach 30% of workloads maybe takes 15 years something like that um because each one of these adoption curves are faster and faster but I don't (20:10) think it's over in that I mean there's still in the reality is like what like the flight booking example the return flight was a mistake or there's lots and lots of these issues or rough edges that exists Within These systems and just the way that we learn how to use different search engines or different kinds of tools to extract what we want there's a big education component about how how and the Improvement of this stuff so where we are today if you look at a modeling you say it's the model size the data and the compute that really drives (20:44) the the pre-training uh and outcome and then youve got reasoning on top of it we where which one of these elements do you think is going to be the driver of the improvements that need to happen for AI to become more mainstream and usable within our organizations because as you said as it is today it's helpful in many areas but it is not the game changer that we know it can be but for it to become that game changer something has to evolve from where we are today but there are limitations in terms of if data was the big unlock well we at a (21:17) data wall and so now it's synthetic data that we need to create and that that tells you a little bit about where it will Thrive and where it won't um but some people say it's not the data it's just the compute the one that has the cheapest compute and the most of it will be the winner in the end and they usually Point towards a Google in that instance what are your thoughts in terms of what's going to lead to the improvements that will make this more usable uh within organizations well I so I think cost is the first one they still (21:47) really expensive to run these things and if you talk to I mean we talk to many different implementers has a different companies the the cost structure of a lot of these models is unsustainable it doesn't make any sense for the product actually so I think yeah I I was tempted to write this post which is like AI is it Groupon era and what I meant by that is you guys probably remember the Groupon errow Groupon came out and then there were thousand Chinese competitors that prices went to zero Alibaba has cut their prices on AI by like 65 85 and (22:18) 97% in three different cuts in the last n months just came out is also that was miraculous right and that so they you know they were able to train a model of 5.6 six million dollars and other to compete with models that were trained in with the hundreds of millions exactly and so I I think there is a hugely deflationary pressure on um on the cost of insurance and that's that's necessary ingredient because the stuff needs to be really cheap in order for it to be ubiquitous uh the the reasoning and the like the amount of training data we've (22:57) trained on all of the internet there's nothing like 20 to 25 trillion tokens of text and uh and then there's a video transcription and that kind of stuff but all of that's been that I mean we're using it just to the extent that we can now we're looking for more data but marginal benefit that data is less the reasoning stuff I think you need to we need a research breakthrough there because it's a different kind of machine learning called reinforcement learning um and there you know you if you our conversations with AI research to say (23:26) like the transformer architecture which is the basis for this dig wave Innovation and AI is everyone's expecting it not to last much longer because it's quite old at seven years for an AI architecture and so people are looking for the next one so I I think you need a pretty s we want a pretty significant breakthrough around reinforcement learning and just to make like um the transform protu is good it could sees many examples of the same thing it can resynthesize an example customize to what you want reinforcement (23:59) learning is a different kind of machine learning where having never seen a problem before it can iterate through a solution pretty rapidly so you need to combine those two to get to what most people call AGI do you think that if you look at it from that perspective do you think that we are as close to that breakthrough as you know there's a lot of conversation happening in the ecosystem right now Sam Alman wrote this blog post about how they know how to create super intelligence today now everybody on the that's against open AI is saying they're (24:35) just trying to raise a new round and this is positioning to be able to go and raise an U round on what is already a massive valuation because their burn rates requires them to raise a lot more money to be able to keep going um but others are saying no like they understand now what has to happen it'll take a little bit of time but multiple people including the anthropic founder has said that we are you know a couple of hundred maybe a couple of thousand days away from AGI or in terms in terms of super intelligence do you think we're (25:06) that close from what you have seen from the outside yeah I think it's really hard to say well what I mean by that is like this whole term of like AGI nobody really knows I mean everyone has there many different definitions of what AGI is if if we're talking about AGI being PhD equivalent in a bunch of different fields You could argue where there right like you look AT3 03 and and it's the ability to solve math problems right but again like you know there are papers there that say just perturb the problem just a little bit and the performance (25:36) ultimately Falls so is it really just anyway so I think it it kind of hinges on your definition but I think we're at the point now where if you wanted a living encyclopedia that could write a paper for you better than most college students we have that we we have that and and then the challenge there is like nobody really wants that so that's to the point of like what's the definition I mean you want that because that's what you do every day but I think if we think of like true AGI as you know her uh or even like Westworld or something like there's (26:17) an earpiece you say book a hotel get me the nicest room it's done you know 5 seconds later hotel room booked you know here's your confirmation code feels like we're still quite away I don't even know if that's AGI that's still just like an admin that's just like an ex a Wasing machine saming machine so I think I think within 12 months you'll be able to do that I don't know if it will work every time but some in New Y piece say book the Ace Hotel in New York City on January 18th to the 20th for a single person and charge it (26:51) to my credit card I think you'll be able to do that well I mean you can kind of already do with perplexities integration with strip and so now it's just tiny ASL are but so this is why this is why I think the the question is Trick is like well what do you mean by AGI that's the most fun thing is like what do you mean if you actually click into what open ai's actual definition is there's a technical definition they have in their agreement with Microsoft which is a hundred billion dollar in profit um that's the definition of AI That's the (27:17) definition of it's a monetary definition in their in their agreement sounds like a venture capitalist definition there's a stipulation that they have which is that once you reach AGI Microsoft doesn't have access to the IP anymore and so when is that point that point is a hundred billion do not in Revenue but in profit if you look at the number of companies in history that have ever got 100 billion in profit there's a small small group right so it's far away um so that's like their technical definition but then obviously you can look that's (27:45) not a very technical definition to be fair though that's not a technical [Music] definition yes. aai enables your CEO subject matter expert and other senior leaders to engage prospects directly creating meaningful connections and boosting response rates with just one click your team can strategically involved senior staff to make your Outreach more credible and effective Empower your GTM team to orchestrate your outbound process and say yes to yes. (28:19) a that's [Music] ye. how does this affect your job as an investor do you think that this flow do you think this impacts the size of markets the ability of certain companies to gain and retain market share I think you know we're moving out of this world or I don't know if we're out of it but you know there was a world of SAS and there was this world that we were in for the last 15 20 years where you could call them Point Solutions but companies like gong Outreach sales left all of the tools that we know at least in go to (28:49) market all the marketing automation tools they could get to 100 200 million in Revenue they many of them have stalled at that point but there was sort of plausibility around getting to a billion dollars maybe penetrating them the Enterprise do you see AI native tools having that same ability and that same because what we've talked about in this show is I bet that there's a very vigorous market for preed and Seed investing because it's easy to see that some of these tools could become could do 20 million 50 million 100 million in (29:20) Revenue but some of the time it's hard to see given how easy is it to how easy it is to copy models and the cost of a or the value of a model model rapidly you know going to zero how easy it is to create new features to the point of you creating an app on Swift you know not in your spare time but like as not one of your main priorities in life so the question is is there do you see the same opportunity for you as an investor for are there going to be the same is the end going to be the same are the same number of breakout companies going to (29:51) exist is it we still see that they're going to be breakout companies but fewer of them or more of them is there a secular point of view that you have on you know your your job is to pick 10x returners like do you see any difference given where we are with AI yeah I think um I think the size of the outcomes will be significantly larger and I think the categories in which startups will succeed will be has initially are the unloved categories of the previous generation so legal software accounting software um stock automation you know these and okay (30:35) what is that well because um so let's why do I make these two points we we look for companies the AI companies at the application layer that have fit into three boxes or check three boxes the first is there's a labor market shortage for whatever re not enough graduates return rates in that role or 100% a year people don't want to do that and that's because in second box the job has toil really repetitive work if there's a certification or training manual or some kind of a test that you can put a human through guess what the AI can do it (31:08) better guarantee it and then the third is you have a hiring manager who is desperate there's some need that they need they need some they need something not someone but something to do this job and so state-ofthe-art is 70 to 75% accuracy for AI for these multi-step processes so if you have a hiring manager who desperately needs to hire somebody for a role that he cannot find that he's probably he or she is probably willing to accept the 75% accurate solution if it costs one six the cost of a human and so that's why like you look (31:39) at paralal work okay historically not you know not everybody's not many people's First Choice as a job right you think about like a s a security analyst who's like reading alerts off email security not and every day it's the same notices and there's just a ton of noise or you know any accounting like a how many times can you look at a tax return and think it's novel it's just it's wrote work and so that's what I think and then this the second part would be larger I do believe I do give some Credence to this idea that AI will (32:12) capture labor spend and it will take longer to achieve it because the productivity gains happen but the iar the profitability doesn't happen immediately so let's let's say we wereing an accounting firm and there's 100 accountants within that firm we adopt AI in 2020 not can suddenly repurpose a third of the organization within 2025 the cost structure should remain the same may even increase see that profitability gain but in 2026 and 2027 the organization will start to change we won't need to hire as many people (32:42) because the bookings per accountant or the number of returns per accountant will subtle increase and over the next three or four years we'll start to see this pretty significant iita um gains or profitability gains and so that's why I think it'll start an unloved historically unloved Industries for software or SAS in the last wave and uh and the outcomes will be significant could be significantly larger so the number I think the like the DECA corn yield the number of deca corns per thousand startups funding will (33:12) increase that's nice I I guess the question there though is that the number of losers and the outcomes on the other side like also go up by the same same number is there like a kind of a balancing of like okay you have thousands of these companies that are started picking the winner becomes much more difficult because the outcome is is is way greater yeah I one of the things and I I think Sam touched on this it's it's much harder to separate it's much harder to create competitive differentiation I think in this category at least early on (33:44) aside from go to market execution because really what you have is the model layer very few companies are now building their own models you can't differentiate there you can build on the promp layer which are these instructions of how to Rend power legal breed or you can iterate and iterate and iterate but again that's just that's just time and experience and then the last is like the uux well I mean realistically like the most sophisticated uiux is a is a box a Google search box but a little bigger and so you can't really differentiate it (34:16) it's on at the application layer it's all on the go to market all comes down to human to human computer human to human interactions can I convince a blue uh whes what they calling white shoe white shoe Law Firm I think that's what they call the fancy law firms anyway can I convince a top Law Firm to adop I've never been in one so I wouldn't know well then there's the green shoe that's part of the IPO anyway all kinds of different color Sho does that scare you as an investor that the competitive advantage that seems to be the big one (34:47) available is go to market execution like humans hand-to hand combat against the market like does that scare you as an investor because in the past maybe the competitive Advantage was a little bit more technical um and it less humans less human variance and a trash involved in that like how do you how are you dealing with that shift so I I would I would submit there has been no shift I would submit that most SAS applications were web apps on databases right Benny off has said this and and now the reality is like you can (35:22) build a web in a couple of hour maybe over a weekend so so like you know the difference technically speaking the difference between two CRM Technologies distribution not that huge distribution it's all distribution right like you read Benny off's book like the key to Salesforce was not a multi-tenant versus a single tenant architecture it was not the Apex language versus a different XML spec or the fact that they were using orle versus my SQL under of it it's the same like are using llama 3 or you using CLA end users in care so I (35:58) I don't I don't know if that's really changed and so this is why you know when people ask likei obviate humans and soft the answer is no because the point of humans in in software sales has always been to engender trust and that I think is extremely difficult for nii to do well we're all over here smiling as we're all in the go to market World we're like okay cool we're secure we have right decision we did it yeah I think you know I mean one startup we met who said that 10 years ago a software buyer was probably at the point they contacted an (36:35) AE was about 10 to 15% of the way through their buyer journey by the time they reached Nai and now with all the webinars and the white papers and chat Bots and stuff the buyer probably at 70% of the way through because they've educated themselves to a much greater extent so I think that will continue to happen right but they'll always be I don't know what you want to call it 15 to 25% where I'm calling Sam and I know Sam and I need a particular I need an Erp and I trust Sam will deliver that Erp so how are you making these bets now that (37:11) let's say you meet five companies that are making the net version of the CRM they're like look at this AI first CRM all this cool stuff that we can do um and so they they're making five similar products AI native interesting could be disruptive especially in the SMB um could really take a lot of Hub spots market share maybe great how do you differentiate at that early stage between these five as to which one is going to be able to have go to market execution at the level to win this category like what are you looking at at (37:38) that point well I mean so at the very very early stage before there's significant metrics the first thing you look for is the storytelling of the company can the company tell a story because okay so one of the big differences one of the things that's common to the early days of SAS and the early days of AI is someone is betting their career on an architectural change that's different right like if I'm moving from HubSpot to Salesforce there's no architectural change I don't have to convince anybody to put the data (38:12) in the cloud sign up with a credit C whatever everybody's done that this is an architectural change and so somebody is saying I am willing to put my career on the line in this company because I believe this is the future and if I do this I will be promoted and if I don't I'll probably be out of a job and so there is there is a start founder within that CRM ecosystem who can who can tell that story so I think that's number one um and probably the most important thing and if you can tell that if you can tell that story and get (38:44) people to believe then then the early adopters will look at it and say like AJ you don't have the product that I want today I know you don't you know you don't I have a road map request list a mile long but I think quotapath will be the winner because I believe you and so I'm willing to make the BET today that in 24 months the product will be closer to what I want and you will be the market leader and then I'll be able to put quar to pass on my resume and that will get me the next G and so I think this is kind of the state of AI sales (39:19) today where you have to convince people that the career risk is worth taking and there again like going back to the previous point an AI a chot will never never do that yeah that's interesting because I think there's a difference between selling Evolution versus Revolution and right now we're selling a revolution um essentially and it's really really hard to do it the other implication of what you're talking about um seems to be on how the cost structure of a tech company might evolve so there's this metric that a lot of people (39:51) in the go to market Community talk about these days which is go to market efficiency and essentially the cost a lot people two people two people I like it D David Spitz created it toas the guy that used to do the Pacific Crest research uh study and David Scott's right hand in all of his data so it's how much money are you spending for dollar of net new ARR and what they're showing is that we just keep spending more and more than we used to and so one of the and so then there are people that are saying we need to try to get back (40:22) and what we've or at least uh something I've been thinking about is maybe we never get back to the go market efficiency of preco um the world has changed too much actually distribution is going to get harder so the cost that goes into winning distribution increases but you find efficiencies in other sides of your business because one can imagine today that in a year or two from now the amount of Engineers you going to need to build a product that is let's say a series a a good series a product you might need half the engineers than what (40:54) you needed today in just two years from now eventually you might need quarter so what might happen is at the p&l level the cost structure changes where your R&D cost comes down your go to market cost goes up but it made sense overall do you see that playing out a little bit that we're going to have to learn how to design these companies and organize them a little bit differently absolutely yeah absolutely I mean one of our startups um he uh this companies basically just on the precipice of of hyper growth and uh (41:25) he he was entering the CEO was entering Q4 10 software Engineers give or take 11 11 or 12 maybe 15 Pro of concept and um he answered the quarter and he said there's no way I can serveice all these he was telling me this over lunch there's no way we'll probably lose half because we cannot address them and then he bought one of his head of engineering said hey I want I want an AI coding tool for the whole so true no problem he got it they'll close like 75 80% of those pox W and then I asked him hey what's your (42:01) engineering head count plan for next year and you know a company will 4X or 5x or more and he said we're not hiring anybody wow and so I think you're right like I mean I have this mental math and are this like a in my head where it's okay you can imagine like when the startup is at zero 100% of the spend is on yeah R&D yeah and then by the time you're like 20 people are a million in ARR maybe like 10% of the spend is on sales and by the time you're 25 to 50 maybe 50% of the this of AR is spent on sales and by the (42:40) time you're at 100 million in AR okay but twox sales and marketing whatever R&D is you might get to the place where you know headcount in R&D teams remains maybe there's 25 50 people for companies generating multi hundred million dollars of Revenue and so I think you're like that's one big you can have the sales efficiency and I don't think you can put that Jim me back in the box I think sales efficiency is monotonically increasing and so the question is okay how do you justify it one is a reduction in R&D expense the other one going back (43:12) to the point we made earlier is the the ROI is better because you're automating more of the job and so the customer is shifting their labor spend to software spend you can charge more oh interesting yeah yeah cuz I we actually for the first time going to click in on one of your predictions which was sound completely hijacked this and then 40 minutes a great conversation it's been fantastic way better than the original plan but there was this prediction of you'll see the first 100 million doll ARR company with 30 or (43:45) fewer employees um in 2025 um and when I read that I thought of this I was like okay how many in sales and marketing versus R&D and I thought of like a 2010 like 20 people in go to market 10 and everything else is kind know where my mind landed up with something like that um it's just really interesting to see this shift that's happening yeah you can imagine maybe if it's a plg company it could be far less right um where you have you have 15 Engineers maybe 10 go to market or 15 and 15 how do you sell a revolution as (44:18) plg like you know when it comes to real sophisticated architectural changes that we're selling to end customers like they even sales even a sophisticated salesperson struggles to get a person to buy a revolution and to put that career on the line and get the buying how do you do that with plg like I've always I've been wondering like the viability of plg for complicated products in this phase shift yeah I I agree with you I I don't think you can sell a complicated products through plg the companies that we've seen grow really fast in plg solve (44:50) a problem that is is a is a Google search query simple So within like uh insurance how do I submit my semiannual state report for me as an insurance broker that I never want to do sign up with a credit card sign up with a credit card half of State Farm is signed up then convert it to an Enterprise account the question with those kinds of go to market strategies is well they might be competed away very quickly right and so the plg strategy I think pretty quickly needs to be complemented with a broader suweet strategy and you could say okay well do (45:29) an AI productivity of Engineers you look at Rippling if you were to build Rippling today with AI I'm much quicker could you build a site and so I think that's that's the risk that you would take both as a builder as an investor which by the way ripling talks about this a lot they have thousands of Engineers and like Parker's conrades talked about their R&D and how how important it is and so it will be interesting to see given the number of products they have like 50 SKS uh and the amount of products that they (45:59) released they just released an applicant tracking system an ATS last month like how fast they're developing what their Engineering in Rd looks like as well uh would be interesting to see um Tomas did you have a company in mind and you don't have to obviously state it when you like made that prediction was there like a company that you've seen firsthand that you maybe even invested in that you're like I think these these guys are going to go all the way this year to get to that 100 million below no I mean I mean (46:27) you know I don't have a company in mind I think it mid journey is probably you know it's up there I don't know if it's achieved that milone already but very very small team with very significant Revenue growth um but no there was no there was no company in Lin when I made that prediction you did this saster talk a little while ago um which was really interesting and the two things that I by the way a little while ago it was two months ago and I rewatched it and it like some of this feels of date already it's so crazy how the world is moving I (47:01) was like watching I was like I don't know if I agree with you're right it was Ooba I think right October two months ago months ago and so there were two or three things one was this reason to not buy AI has shifted from security to Roi which I thought was fascinating the second was you spoke about the impact of extended sales Cycles on a on a company and I think it was really fascinating to see if a company sales cycle increases by 50% du to an example of a company whose burn then increased by three ATS and that's disastrous um and then the (47:33) fact that you were speaking to some C who had implemented AI who was seeing certain signals that it's working on their sales team I think they were generating more pipeline but then you asked them how much of that pipeline are you closing and the answer was nothing um which was also really interesting which I think comes down to your point of AI is not really driving a productivity increase and go to market just yet has your thinking around these three elements changed at all like do you think sales Cycles will continue to (48:03) increase in 2025 do you think that um the reason to buy is going to shift from security to something else sorry to from Roi to something else and do you think that AI is going to start showing a result and this is more about we haven't learned how to get the result um or something else yeah I think I think we haven't learned how to get the result that we want and then once there two or three companies in a category that demonstrate how to do it everyone else will buy the software and the sales Cycles will collapse because they'll be (48:35) such a fear of massive share shift Oho yeah and and so then then it kind of collapses I I think the overtime and we've seen a thousand X reduction in AI cost over the last two years and so I think we'll probably see like another 100 to a th000 x in 25 and over time and also like the use of small models and open source models that will help so I think I I I think if if I had to pick the most important one that we would solve as an ecosystem to see massive growth it would be the last one which is people users are now sophisticated (49:12) enough or the software takes the user input and then figures out how to manipulate the AI to produce what they want if we could solve that problem then your sales Cycles I mean you're talking days because there's just no way a sales leader or a marketing leader will watch a competitor massively scale content or ads or uh account plans or use customer engagement Prospect engagement and and stand dly by Sam you have the operationalization of AI and go to market as one of the themes for Pavilion CEO Summit and maybe even (49:47) just for the entire year um how are you thinking about people's openness to share all these things I I've been thinking about this idea of is the ecosystem going to become more secretive now so product Innovation isn't defendable so we try to like actually not publicize it the same way we're noticing this with models right now where they haven't really published what sora's like technical size Etc is they holding more information to themselves will we see that happen across the ecosystem um in terms of product (50:18) releases and the usage of different tools we don't try to hold on to that or do you think people will openly share stuff like they have in the past what are your thoughts on this I think it's a great question and I think I I don't know about openly Sher but I think market dynamics will force Innovation out into the open and into the public domain so you know ganto and Gom uh I guess his nickname is G I forget his last name I think it's carbon but he um he was like the interim CMO for ramp and was one of the people that helped it was (50:49) like one of the fastest growth rates to a 100 million in ARR in you know the last 10 15 years and he's now part of a consulting firm that teaches people how to operationalize AI in their goto market and I think that that's there's a lot of pontificators and people that don't know what they're talking about but there's a lot of very hardcore practitioners and there's a lot of conversation about it and I think both communities it's communities like Pavilion but also many other communities that are sh that are sprouting up on (51:16) Discord on slack on WhatsApp and just informally where people are sharing how they're deploying AI how they're architecting their teams there's this company cargo and this guy um I forget his last name but the CEO and founder of it who has coined this term GTM engineer as it would replace sdrs and like that's what the new world is and it's somebody that can use clay and use all these automation tools and data enrichment tools to build custom workflows that can do exactly what Tomas was just talking about so I I don't know that because I (51:47) guess one of the points that I'm making is I don't know a lot of the people that are going to be great at this are going to work at companies I think they're going to be Freelancers and a good point Mercenaries that because I think people are so desperate for how does it actually work and I think again to the point of why Pavilion exists I think the idea that I don't know sales talent agency no offense but it's going to be the place where you go to become an incredible GTM engineer as a as part of your job in demand generation for sales (52:17) talent agency I think it's just highly unlikely I think that what you're gonna have to do is talk to other people that are doing it I think it's going to be constant practitioners constant you know sharing their what's working what's not working so even though there might be like big Enterprises that are very protective about how they do this I think one way or the other uh people are going to be talking about how to implement and operation plugged into those communities great point I was talking to the CEO of DBT Labs Tristan (52:43) um here in Philadelphia that really could talk about communities and uh customers just coming to them and six months ago they were just saying like we have money to spend on AI like how do we actually spend it they didn't have their for point of view and I caught up with him the other week around that and it's like hey how's that going because that seemed like just such a huge massive opportunity that you have just built this like ecosystem um and Community centered around your product like it's a really really big opportunity like yeah (53:13) we've been working like that's where we spend all of our time day and night on how do we operationalize this and bring this to Market but the thing that Tomas said earlier which is like where the most gains are to be made is in Consulting and services so it's interesting to see these AI services and Consulting practitioners pop up but also at the same time that's the area where the most disruption probably will happen in 2025 as well so they're kind of like I don't know if they're butting heads to I don't know if you you think about that (53:42) or still it's still the point AJ of trust you know that's like the point that like is McKenzie gonna be dislocated no probably not because one way or the other you're still making a big corporate decision and even if McKenzie's EA has skyrocketed by 10x because they don't have the whole Junior layer of people out of school because they're using AI to do all the research and all the strategy work you still need this is why brand is ultimately like one of a very meaningful competitive how many different companies are going to be (54:13) teaching AI for go to market courses online courses I meet somebody every day that's going to be doing it but when I do it there's a much bigger brand behind it and if and of course if I'm competing with Wharton and they've got a bigger brand than me and so I feel like that's brand equals trust and trust is is how you reduce sales Cycles improve R rates so then the question is how do you build a brand and and again that that's not easy and that doesn't that's not cheap either that's not it's not getting any (54:42) cheaper toas do you think that this this the services industry that is this huge Market when you Loop all of services in but it's not one type of services right you've got this got McKenzie you've got Tau at consulting services and these are fundamentally different types of businesses one is labor Arbitrage for simple stuff you're just essentially saying I don't want the p&l headache of this cost so you take the p&l headache I'll give you a little bit of margin whereas when you're hiring McKenzie you're not necessarily hiring them for (55:15) that reason so these are very different in terms of the sophistication of the service that they provide how do you see AI impacting these two types of services company like is the is the near-term impact going to be on the Tata consulting services type of firms the RPO agencies like recruitment process Outsourcing agencies and then the McKenzies are going to benefit from the productivity uh enhances that they can get but the core of what they do still is defendable or is it not what are your thoughts well I mean I think it was (55:49) Accenture generated three billion of Revenue in the first nine months of 24 and that made them the single largest producer of AI Revenue in the world beating up I mean Google whatever setting aside the infrastructure companies right so uh whatever we'll call them top 10 so I think just in terms of like lift to those businesses they those the big Outsourcing shops um they will benefit hugely if they're on The Cutting Edge and our understanding is they're all the major ones are either have individual teams or subdivisions (56:24) where people are pushing pretty hard they're partnering with startups take advantage of that so I think in this case actually it's it's the larger companies who will benefit more than the smaller companies I think this and I wonder been debating this in my head but I think this might be the like late Motif for AI overall which is the big companies's initially benefit on the first wave of interest in AI the smaller companies are then forced to really innovate and differentiate and they come up with something that's pretty (56:54) meaningfully different and then as a result in the second wave ultimately overtake oh that's interesting so in this Theory like a lot of people have been saying 2025 is the year where Enterprise SAS companies the publicly traded ones are going to get demolished um but in this instance they'll actually be the ones to first get the lift and then eventually that'll give some time to I don't know like a day. (57:19) AI to get enough momentum to then attack them a year or two down the line is that yeah I I think so because I think okay if you're if I'm sales if I'm Mike Ben off I'm not going to innovate on the models you know I'm going to sell what the customer wants today I'm going to sell like education base level CRM automation I will tack it on or integrate it to the extent that I can with my existing architecture and then there'll be a founder of someplace who says you know what that architecture no longer makes sense in this universe I'm building I (57:55) mean it's like the Uber app right the first time I used location that was native on the phone never wanted to go back and so I think that's what happens and and that founder will need to figure out a way of stitching together I don't know five or six different models and completely reimagining the workflow where you is a calcified Beast is focused on the bdr the AE the CSM motion that you um and all of us GRE grew up on but that motion well won't work anymore right this doesn't because the SDR is managing 50 email accounts the AE is (58:27) processing twice or three times the number of leads and so the existing workflows the pipeline analysis that Clary or others have built May no longer be relevant because the underlying workflows have completely changed that is very interesting yeah and I guess that is a a good place to end this and move into Sam shout outs right we can do shout outs so we can also do uh do you have another question last week was pretty good normally at the end of the episode Tomas we do shout outs and wins of the week but then last (59:00) week Aid said what are your what is it resolution for 20 what is the one big change you would want to make in 2025 so maybe we'll ask you that toas what's the one big change you're gonna make in 2025 yeah so I am every Sunday I'm writing down I'm looking at my calendar and I'm figuring out what could I have automated with AI last week some part of what you do sounds fun and some part of it sounds exhausting I have how much do you work toas like what's your average wheat like like how many hours do you work what's it I do (59:35) work early in the morning like yeah how do you spend your day yeah it's um uh it's not crazy hours I mean I wake up at like 5 or 5:30 work out breakfast and then what an A's going to bed at that time actually that's his bedtime well but I go to bed at like :30 my day is just gied earlier right I'm not trying to be a jerk so you some people are night hour and some people morning birds or whatever you want to call them I just I should like more so you sit down every Sunday and you look at what could I have automated (1:00:11) the last week and then you try to do that for the next week yeah I mean let just take 45 minutes and figure out okay let's pull up an AI coding assistant can I transcribe podcasts how long will that take right so I need the URL for the toline podcast and ask Thea I go through each of the emails then figure out how to transcribe it just iterate iterate iterate was that your Sunday automation task is that what you did on Sunday I've been yeah I've been working I've been working a little bit on that one just try to understand like and the (1:00:42) hard part actually is not the hard part is not the coding which is surprising the hard part is the prompt which is what do you tell the AI that you want yeah and you can ask another AI you can go to Claude and say build me a prompt for this particular model to produce these following things and that's a really great stock and they tend to be 500 Words which kind of blew my mind right because we trade in the Google era of fewest keywords possible but with Aon if you really want a project beef you want a p a project requirements document (1:01:12) wow a really great Insight because you're right in in the era of Google you're sort of taught like the order of the words doesn't matter there's a lot of redundancy shorter is better specific the order of the words don't matter like you don't need to make sense in the traditional sense no you yeah I found voice to be great for prompting like if you go onto voice and you say I'm just giving you the prompt right now and so we're going to talk through this once we're done I'm going to move back to like the test mode and it's like okay (1:01:42) great and I find that I for me it's easier to give it a complex and complex prompt over voice and then move into the written format the chat format to then work through whatever I want to work through with it that's it's so much more human that's how like if you were to hire my way through it yeah I mean you're a CEO that's what you do all day yeah meand his way through it that is that is what Austin does all no he's briefing people with voice he's telling people this is the project that I want right and then those those (1:02:13) people are collaboratively they assemble a document and then they agree on that and then that person is delegated with a work and executes the word yeah yeah that's true so Salon AJ question for you guys what is one change in ter one new way of using AI one change in how you're using AI um in 2025 maybe based on this conversation maybe based on something you did over the weekend I don't know but any I mentioned this I think uh last week or two weeks ago but one thing I've done is I take all my weekly investor (1:02:44) updates and all my weekly company updates and I upload them into one instance of chat GPT and I use that as basically like a highle strategic interface to have conversations about where am I off base where are things not aligned really I'm trying to look for alignment and then I think who somebody told me I forget who it was create two personas and have them talk to each other so I had my conservative board member and then my optimistic person who's my dog uh like have a point of view on if Pavilion does really well and (1:03:15) if it doesn't do really well and I created a dialogue between them but based on this conversation one of the things I want to do is just is much deeper prompts I want to take the time to think through and I'm probably going to use to tomas's point I'm probably going to use an AI to give me thoughtful prompts that I can then choose between so that I can get more sophisticated answers because I've been you know it's been like a very back and forth piy conversation but not as robust as uh as I want it to be so deeper more (1:03:43) thoughtful prompts that's what I'm gonna do AJ I like that AJ well to I have I have uh one by Tomas do you find when you write prompts that you like assign a Persona to The Prompt like hey imagine you're a Harvard MBA grad that's thinking like this that's the beginning right like you're an AP US English teacher you're grading papers I find that the most annoying part of it by the way it's like it should just figure that out like if I'm asking you a question figure out what mindset you need to be in to give me the (1:04:18) best answer to this question so I mean to that point I think if you're building applications AI applications this will be a big part of the app because uh what's not what's what's uh many of these applications they take your prompt and rewrite it and then send it to AI yeah uh so my sh it's both a shout out and an AI is a tool that I've been using I was introduced to last week is cove. (1:04:50) which is pretty interesting it's a unique look at everyone going and typing it in right now I see it uh it's it's actually a unique way of AI on the application layer that I hadn't seen it it um the use case that they come up with is planning a trip but it creates all these cards it's has a plg so you can all like test this out I have no like investment I don't know who the founders are but it's pretty cool uh easy to sign up and you can type in a prompt and it gives you all the cards and within the cards there's ways to like change the AI I'll get ask for (1:05:19) inputs like you're taking a trip to Miami for the Orange Bowl for Penn State here's the links to the pre-a party you should go into if you want to have dinner with your friends give me the dollar amount and I'll type it in it gives you like a table for the agenda and like we all use chat GPT or Claude or gemini or whatever for travel this is like the use case I think there's a bigger workflow opportunity so talking about workflows and how J works pretty cool out to uh to cove. (1:05:51) this is the interesting thing like this was the fathom stuff right they all features and features dat it doesn't matter it's brilliant but in seven minutes some dude notion is BU you I think you should do is pay influ you get distribution scale we talked about this listen to the episode Austin we got there already okay I'll give thank you so much oh go ahead find found I'm sorry so I don't know how to cook um to the point where like if you asked me to fry an egg to save my life I wouldn't know how to do it um and so over this break I decided to use di to (1:06:26) cook a couple of meals for Sher and so this is the way that I used it I I was at the gym on a bike I put voice mode on and we came up with a menu and then I went into Whole Foods and it was in my year and it helped me shop because I don't know where you find breadcrumbs it's like go in this aisle look up there like it was really interesting so throughout the shopping experience it was in my year and it made shopping a lot of fun I must have looked like niss but it was a lot of fun to shop and then I came home and it was just in my year (1:06:58) as I cooked the meal and I went from not knowing how to fry an egg to cooking a meal with multiple different components to it there was a salad there was a there were brownies there was a steak in the net meal there was scorns a cake um saffron risoto and a salmon that had been like baked and everything from scratch no mites or anything I don't know how to do any of this like I didn't even know you have to Mi the thing and it was just it was a her moment for me where I was Neo in The Matrix that was that was wild like it blew my mind when (1:07:35) now I'm exited I'm like Sher what do you want to eat like I want to go cook something new because I now know how to do it there's something in my ear that tells and I'll be like hey I don't have this thing I don't know how to measure the heat of a steak it's like use your finger poke it like this it was just it was so perfect that's so cool oh and then I had AJ giving me some tips in the background as well a was my you're come on human tips that that's really cool a little bit of tips a little bit of that's my use mostly but thank you so (1:08:04) much for joining us uh thanks for being a supporter of TopLine in Pavilion for many years and uh we're excited to cheer you on and hopefully help you support the companies that you're investing in uh feelings Mutual have a great 2025 everybody thanks for having me on the show thanks [Music] tomor this episode was brought to you by airall the busy Communications intelligence platform deliver a better experience for your customers and your teams at airall that's a i c . (1:08:38) io save time save money and improve performance with a single communications [Music] platform thanks for listening to Topline new episodes come out every Sunday if you like what you heard today subscribe so you never miss an episode leave a five-star review and share it with your friends don't forget to join our Topline slack channel to connect with us and discuss the topics we cover with other listeners click the link in the show notes lastly if you're interested in joining Pavilion you can learn more about membership at join pavilion. (1:09:15) jpg [Music]